CapRL: Stimulating Dense Image Caption Capabilities via Reinforcement Learning
Authors: Long Xing
Xiaoyi Dong
Yuhang Zang
Yuhang Cao
Jianze Liang
Qidong Huang
Jiaqi Wang
Feng Wu
Dahua Lin
Venue: ICLR 2026 Poster (2026)
Keywords: Image Caption
Reinforcement learning
Large Vision Language Model
Primary area: foundation or frontier models, including LLMs
TL;DR: We present CapRL, an effective decoupled two-stage training scheme with verifiable caption reward to boost image captioning model.

Abstract: Image captioning is a fundamental task that bridges the visual and linguistic
domains, playing a critical role in pre-training Large Vision-Language Models
(LVLMs). Current state-of-the-art captioning models are typically trained with
Supervised Fine-Tuning (SFT), a paradigm that relies on expensive, non-scalable
data annotated by humans or proprietary models. This approach often leads to
models that memorize specific ground-truth answers, limiting their generality and
ability to generate diverse, creative descriptions. To overcome the limitation of
SFT, we propose applying the Reinforcement Learning with Verifiable Rewards
(RLVR) paradigm to the open-ended task of image captioning. A primary challenge,
however, is designing an objective reward function for the inherently subjective
nature of what constitutes a "good" caption. We introduce Captioning Reinforce-
ment Learning (CapRL), a novel training framework that redefines caption quality
through its utility: a high-quality caption should enable a non-visual language
model to accurately answer questions about the corresponding image. CapRL
employs a decoupled two-stage pipeline where an LVLM generates a caption, and
the objective reward is derived from the accuracy of a separate, vision-free LLM
answering Multiple-Choice Questions based solely on that caption. As the first
study to apply RLVR to the subjective image captioning task, we demonstrate
that CapRL significantly enhances multiple settings. Pretraining on the CapRL-
5M caption dataset annotated by CapRL-3B results in substantial gains across 12
benchmarks. Moreover, within the Prism Framework for caption quality evaluation,
CapRL achieves performance comparable to Qwen2.5-VL-72B, while exceeding
the baseline by an average margin of 8.4%. Results validate that our CapRL effec-
tively trains models to produce a more general and accurate image descriptions,
moving beyond the limitations of traditional SFT-based image captioning models.

Published as a conference paper at ICLR 2026
CAPRL: STIMULATING DENSE IMAGE CAPTION CAPA- BILITIES VIA REINFORCEMENT LEARNING
Long Xing 22 Xiioyi ; Yuhang Zang'. Yuhang Cao².
Jianze Liang", Qidong Huang". Jiaqi Wang 2,4,1 Feng Wu', Dahua Lin
l University of Science and Technology of China, "Shanghai AI Laboratory,
TThe Chinese University of Hong Kong, *Shanghai Innovation Institute, AAlibaba Cloud, 6 CPII under InnoHK
x xing_1 Jon@maal .ws.u..ceee
Model Data: Capki, Huggi ngr ace Collect
Code: CapRL Githup Repository
ABSTRACT
Image captioning is a fundamental task that bridges the visual and linguistic domains, playing critical role in pre-training Large Vision- Laagun-angguge Models (LVLMs). Current state-of-thh-an art captioning models are typically trained with Supervised Fine-Tuning (SFT), a paradigm that relies expensive, non-scalable data annotated by humans or proprietary models. This approach often leads to models that memorize specific ground-truth answers, limiting their generality and ability to generate diverse, creative descriptions. To overcome the limitation of SFT, we propose applying the Reinforcement Learning with Verifiable Rewards (RLVR) paradigm to the open-ended task of image captioning. A primary challenge, however, is designing an objective reward function for the inherently subjective nature of what constitutes a "good" caption. We introduce Captioning Reinforce- ment Learning (CapRL.), a novel training framework that redefines caption quality through its utility: high-quality caption should enable non-visual language model to accurately answer questions about the corresponding image. CapRL employs decoupled two-stage pipeline where an LVLM generates caption, and the objective reward is derived from the accuracy of a separate, vision-free LLM answering Multiple-Chote Choicehh Questions based solely on that caption. As the first study to apply RLVR to the subjective image captioning task, we demonstrate that CapRL significantly enhances multiple settings. Pretraining on the CapRL- 5M caption dataset annotated by CapRI-3B results in substantial gains across 12 benchmarks. Moreover, within the Prism Framework for caption quality evaluation, CapRL achieves performance comparable to Dwwn2.55VLL-72.
while exceeding the baseline by an average margin of 8.4%. Results validate that our CapRL ef- fectively trains models to produce more general and accurate image descriptions, moving beyond the limitations traditional SFT-based image captioning models.
INTRODUCTION
The image captioning task (Karpathy & Fei-Fei 2015; Vinyals et al. 2015), which generates a natural language description for given image, bridges the gap between the visual and linguistic worlds. The captioning capability is fundamental to various applications, including lasion-language models like CLIP (Radford et al. (02T), which learn a shared embedding space for images and text. Furthermore, captions are often a core component in the pre-training stage of Large Models (LVLMs) (Liu et al. 2023), where the model Iearns to align visual information with linguistic descriptions massive scale before being fine-tuned for other downstream tasks.
Given the importance image captioning, there is a strong need for captioning models that can provide dense and accurate descriptions. Most modern captioning models Chen et al. (2024b);
"Equal contribution. Corresponding author.

---

Published as a conference paper at ICLR 2026
(a) Subjective Caption Reward
Cuption
LVLM Judgo d
Various Inherent Blas
Low-quality Caption
General LVLM
Prefer Verbositty
My description well- organized, well-structured...
Lengthy Inrelvvant
Reinforcement carming
lacking
Outpul overall score caption based on the image.
Reward
Unified Reward Model
Prefer Brevity
Reward
Hacking
animaals this image.
Brief Incomplete
Objective Caption Reward (Ours)
CapRI: Verifiable Reward via Decoupled VQA
Dense and Comprehensive
Caption
Reinforcement
How
there the imuge?
O Hard
there tall red
Leariing
Answer the multiple- choice question on the cuption.
0, otherwise
Hack
wearing light yellow short-sleeved dark green He has sly but cheerful expression and holding
Reward
Shuflle the options average the reward
popsicle...
(c) Training Curve Caption Quality Comparison Reward
Caption Length Token
CharQA
ClarqQ
Math
1500
Verbose wilhoou adding
Reward model was rapidly backed
image details
1250
10.8
Info VQA
1000
Math
00.5
0.4
Traininn collapse
200
Only autpui "description"
MMMU Pro
SEED
200
400
800
3000
1200
200
401
milc
BC0
1000
1200
CharXiv
MMStar
Training Steps
Training Steps
Performance in Prism Framework
Qwen2 5-VL-3B 3B Judge
CapRL
Unified Reward Model Judge
Original Qwen2.5 5-V-2.5-VVL3B
Current LVLM-as aaa--a----juddee jus-a-juggerrwward models suffer from limitations like rewarding verbosity or brevity, leading to low-quality captions and reward hacking. (b) Our CapRL: CapRL. uses a decoupled two-stage VQA approach to provide subjective rewards for captions. (c) CapRL's Advantage: CapRL outperforms previous subjective reward methods, as shown by training curves and higher performance in the Prism (Qiao et al. 2024) evaluation setting.
Figure (a) Existing Reward Models:
Rotstein et al. (2024); Vasu al. (2025) are trained based on LVLMs using Supervised Fine-Tuning (SFT). While effective, SFT requires large datasets annotated by humans proprietary models, which are expensive and not scalable. Furthermore, image captioning an inherently open-ended problem, where a single image can be accurately described by wide variety of captions. Since SFT models are trained to match a single ground-trute description for each image, they tend to memorize specific answers rather than learning the underlying concepts. As result, the SFT models become less general and struggle to generate the diverse range of valid captions possible for a single image.
The limitations of SFT have led to recent paradigm shift in the post-training of LVLMs toward Reinforcement Learning with Verifiable Rewards (RLVR) (Lambert et al. 2024). RLVR is the paradigm that trains models by providing clear and objective reward from the verifier, such a binary signal of correctness for mathematical reasoning (e.g., DeepSeek-RI (Guo et al. 2025)). Unlike SFT, which teaches model to mimic single ground-truth response, RLVR encourages the model to generate more diverse and robust outputs that meet the verifiable criteria. Our objective is to design powerful and scalable RLVR training paradigm for the image captioning task generate more creative and more general variety of accurate descriptions.
However, applying RLVR to open-ended tasks like image captioning is challenging, primarily due to the difficulty of designing an objective reward function. A good caption can be subjective, with multiple valid descriptions possible for the same image. As shown in Fig. (a), these early studies fail to provide accurate reward signals for RL training. Using reward models (Liu et al.) (025b Su ctt al. 2025 Lu 2025 or ALM-as-a-judge (Gunjal et al. (025) to provide feedback is vulnerable to reward hacking. The captioning model learns to exploit weaknesses in the reward models (e.g.. verbosity or brevity outputs) rather than producing a high-que gigh--uulity response. Moreover, it difficult to create effective rubrics evaluation prompts for LVLM-as a--------juuee methods because captions free-form and encode substantial information. Using reference answer as rewards (Gurung Lapatal
2025
Yu al. 2025) like ROUGE (Lin 2004 and BL.UU (Papineni al. 2(02) is constrained
2

---

Published as a conference paper at ICLR 2026
Enhanced Structure, Coverage, and Accuracy
kulls sediunge seee 2011
Orginal QwenZ.5V SVL-3B
Afler CapRL:
(right) income tiess The peresntages are gives follows
The inceme liess, foom lowest to highte, are classifiia is "Lower, "Middle" and "L Upper"" Ench har highlights the percontage of adults living income fier for the reppeative
Lower [Mdddle Upper
ther, "25%g" "26%, and *29%, veerically aligned for each categuey. Dease
The zumbers the indicate the
adults
2011 29%% 51%
2016 29% 552% 19%
25%
20%
10
lifecme changes in the share of acults living in middle income
graph indicates
198] 20% 59%
27%
17%
18%
householss...
1971
[255 611
14%
Fewer Hallucinations and More Details
Original Qwen2 SVL-3BB
AAter CapRI:
-- The main focus on man dresed a gny sait, innsfarinn frring on envelope or rificee a women white diess The envelape is ficmal dacumeni, which ofler signlliaatt at weddings The couple being distracted scens be the primary colebrante this clip. persen lunding appears the midcle making way through the giests..
TThe bride is wearing white dress and is langhing heartily, Iuildnng las beuquet, She lus carly blande hais and spetta to be in her mid 50s to 40s
OOn the left is
in pnay wiil, likely greams weoring
Beis the laughing and appears
enoclope gloesss and
holding woman by army suggoring he her huebard The grusen wearing is cheoiic gray sull, presenting a formal appezrance typical of wedding attine..
colebratory
Hallucinations!
Figure 2: Illustration of the captioning capability improvement CapRL brings to Qwen2.5. VL-3BB.
when evaluating complex and long-form captions. Fig. I (c) further demonstrates the limitations of previous subjective caption rewards, showing reward hacking and unstable training curves.
To design the objective RLVR reward function for the subjective image captioning task, we introduce novel perspective, where caption's 's quality is proportional to its utility. When the image caption is detailed and accurate, a text-based LLM that can't 't directly "see" the image can still answer Visual Question Answering (VQA) questions about the image, For example, for the question "What color is the frisbee?", the LLM finds the phrase "red frisbee" in the caption and correctly answers "red." " Driven by this motivation, we present an effective decoupled two-stage pipeline, dubbed as Captioning Reinforcement Learning (CapRL), as shown in Fig. (b). Specifically, the reward of our CapRI. framework is determined by how well a caption generated by an LVI.M enables a separate non-visual LLM to answer Multiple-Choice Choice Questions (MCQs) about the source image. The L.M'' resulting accuracy serves as the objective reward for the RLVR training. To ensure the high-quality MCQs data that present enough knowledge required for VQA has been examined, we also developed a specific QA curation pipeline. The images are sampled from various sources, including natural images, charts, and documents. The questions and answers are filtered to ensure the questions can only be answered by analyzing the image content itself.
We conduct comprehensive evaluation of the significant benefits brought by CapRL. From qualitative perspective, as shown in Fig. 2 applying the CapRI. framework to Qwen2.5-VI--3B makes its outputs more well-organized and accurate. Further illustrative cases for various charts, infographics, or natural images can be found in Appendix A From a quantitative perspective: (i) We employ CapRL-3B annotate the CapRL-5M caption dataset, and LVLM pretraining on this dataset yields substantial improvements across 12 benchmarks. (ii) Furthermore, using the Prism Framework Qiao et al. 2024) for caption quality evaluation, we observed that CapRL-3B remarkably achieves performance comparable to the 72B model, and outperforms the baseline by an average margin of 8.4%. These results demonstrate that our CapRL. framework, by leveraging objective reward design as reliable optimization signal, effectively drives the model to produce dense and accurate captions.
Our contributions are summarized as follows:
1) We contribute the first study of applying Reinforcement Learning with Verifiable Rewards for the open-ended and subjective image captioning task. Unlike traditional Supervised Fine-Tuning, which can lead to models memorizing limited set of annotated captions, our method allows the model to explore and generate a broader range of creative and general descriptions.
2) We present CapRL, new training paradigm featuring decoupled two-stage pipeline. The initial stage uses LVLMs to generate rich and accurate captions. Subsequently, the second stage evaluates caption quality by using vision-free LLM to perform the QA task. We also created specific QA curation pipeline to ensure the quality of the questions and answers used for the second stage.
3) We carry out extensive experiments to verify the effectiveness of CapRL. Notably, both in the LVLM Pretraining setting for modality alignment and the Prism setting for caption informativeness evaluation, CapRI. consistently exhibits superior performance compared to the baselines.
3

---

Published as a conference paper at ICLR 2026
(a) CapRI.
Verifiable Reward
Policy Gradient
LVLM
Optmmzzttio
if correct
Shuffle options
Referencc Model
otherwise
L.M
Avoid potential bias for unswer choices.
KL loss
Average reward
[+ Caption
This image from the animated movie Zootopia. shows three main characters in comical and memamble scene that... police officer, lrughing energetcaall.....
Judy Hopps, enthusiastic
Answer the question based on the caption.
Policy Model
Describe this image (b) QA Curation
Sample times
Filter Questions
Datasct
Who laughin in the image?
Q1
July i Nack C.ilesh
Aveid Leakage
D. No ase
Generate mulripla-choice questions answers bowed on the imagee
uninalls
75k
Q2
image?
questions related image
QA pains
Figure 3: Overview of CapRL. Unlike the standard single-stage RLVR, CapRL performs decoupled two-stage process. Captions generated by the LVLM, paired with curated MCQs (b), are used query LLM, whose resulting accuracy becomes the objective reward for the LVLM (a). Our CapRL offers a scalable framework for applying RLVR to the open-ended image captioning task.
2
RELATED WORK
Image Captioning.
Early Large-scale image text corpora (Schuhmann al. 2022 Changpinyo
texteetexttee
ct al. 2021 Thomee et al. (016) have driven iision-languee pretraining. To scale and improve
captions, researchers design advanced captioning pipelines: BLIP-LATON (Li et al. 2022) generates short synthetic captions, LaCLII (Fan et al. 2023) uses ChatGPT to rewrite them, and CapsFusion (Yu et al., 2024) consolidates and refine information with fine-tuned models. Besides, there are many research projects which use GPT-4V and buman-in-the-lon
pipelines to produce richer, fine-grained annotations such ShareGPT4V (Chen al. 2024b) and ALLaVA (Chen et al. 2024a). Recent studies Li et al. 2024| Sun et al. (2024) have explored multi-expert approaches to compensate summary, some works rely on complex pipelines with multiple models, training- free but costly at inference, while others require lots of expensive labeled data for SFT. In contrast, our CapRI, achieves strong performance with remarkable data efficiency through RLVR.
for LVLM limitations.
Reinforcement Learning with Verifiable Rewards (RLVR). RLVR (Lambert et al. 2024 rep- resents a promising paradigm for training Large Language Models (LLMs) to tasks that have an objective, easily verifiable reward signal. For example, in mathematical problem-solving, the reward can be a binary signal of correctness (Shao et al. 2004). and for code generation, it can be whether the code passes unit tests (Team et al.] 2025). Compared to the traditional Supervised Fine- Tunnng (SFT), RLVR offers a more robust and scalable approach. While SFT trains models to imitate a set of provided ground-truth answers, often leading to models that memorize specific phrasings (Chu et al.) 2025), RLVR encourages the model to explore and discover optimal solutions. This is particularly beneficial for problems with multiple valid answers or reasoning paths.
METHODOLOGY
An overview of our CapRL is shown in Fig. The CapRL framework consists of novel, decoupled
two-stage process. In the first stage, an LVLM generates a caption for an input image. In the second
stage, this caption, along with series of MCQs, is provided as input to an LLM. In the following, we
will describe how to apply RLVR on the image captioning task via our CapRL in Section 3.1 Then we use the model trained with CapRI. to construct the CapRI--5M dataset in Section 3.2
3.1 CAPRL
The design of the reward function is pivotal factor in the success of RLVR-based approaches, since the reward function directly guides the optimization direction of the policy model. Although
designing reward functions for objective tasks (Shao et al. 2024 Liu et al. 2025c
Luo al.
4

---

Published as a conference paper at ICLR 2026
2025 is straightforward, developing the reward function for the subjective image captioning task is
challenging. While reward models (Liu et al. [025b] [Su et al.) 2025 Lu 2023) or the "LLM-as-a- judge" approach (Gunjal et al. 2025) have been explored for RL traming on open-ended tasks, these
models are still vulnerable to exploitation in captioning task, primarily owing to their intrinsic biases, which may unintentionally encourage the captioning model to produce verbose or brief results.
To design a reliable verifiable reward module, we leverage a perception-reasonin task as a proxy to evaluate the quality of captions. The overall process of our proposed method CapRL, is illustrated in Fig. During the GRPO training process, an image and an instruction are first provided as input to the policy model sample a set of candidate captions. Each caption is then paired with corresponding questions and fed to a Large Language Model (LLM). We assign each caption a reward score based on the accuracy of answers generated by the L.I.M. Subsequently, we calculate the mean and variance of rewards across the group to derive the advantage for each caption. To ensure training stability, and consistent with the original GRPO framework, we incorporate a KL-divergence penalty. The policy model is then updated via policy gradient optimization.
decoupled VQA
To prepare the data for GRPO training, we constructed a VQA dataset composed exclusively of multiple-choice questions. This multiple-choice format facilitates the computation of verifiable rewards. Throughout this curation process, we utilized an LVLM to filter the data and prevent data leakage. Further details regarding our reward design and QA curation are provided below.
Reward Design. Specifically, given an instruction and an image, the policy model My generates set of captions {cI, (c,. .,,...; c.). ). Each caption is then paired with questions related to the image and passed to a large language model (LLM), denoted as MM,, for answering. Since the MM does not have access to the image directly, its ability to answer the question correctly depends entirely on how comprehensive and accurate the caption is. Captions that include more relevant objects and detailed descriptions, are more likely to provide the necessary information for the LLM to answer a question correctly. In contrast, less informative captions are more likely to lead to incorrect answers. Since LLMs exhibit high stability in answering multiple-choice questions, and the evaluation of their responses only requires exact matching, the accuracy of the LLM's responses can therefore serve as reliable indicator of caption quality. This
process can be formulated as:
am Mr (G, fm),
where Im denotes the m-th question associated with current image L, and am is the LLM's answer to that question. Then the reward for a single question is computed using simple exact-match criterion:
1, if am GT
r(am)
0, otherwise.
Here, GTm m is the ground-truth answer to question Zm.
To eliminate potential bias in the LLM's preference for specific answer choices, we randomly shuffle the options each time question is presented. Additionally, relying on single answer evaluate a caption lacks robustness. To ensure the stability of caption scoring, we sample N times from all the questions related to the image and let Mr answer them independently. The final reward for a caption is computed as the average accuracy over these N sampled questions. Formally:
Reg Σ r (Mr(cs, Shuffle(am)))). mh (L...,, MJ.
Here, M denotes the number of questions associated with the current image I. Since we compute the caption reward directly from the original caption, there is no need to perform intermediate reasoning steps as DeepSeek-Re R1,, which first carries out thinking process before formatting an answer. As result, our method avoids the need for any format-specifie rewards and retains a clean, flexible reward computation process that fully respects the free-form nature of the policy model's output. is important to note that, in our GRPO training setup, Qwen2.5-3B Instrrctt is used as Mr by default, which makes the overall training highly efficient.
QA Curation. To train CapRL effectively, high-quality VQA dataset (g. a) with question q and answer a is required to provide reliable reward signals. We construct this VQA dataset using a structured three-stage curation pipeline. (1) Image Collection. We begin by sourcing diverse images from the web and existing open-source datasets, including natural scenes, charts, and documents, to maximize variety. (2) QA Generation. For each image, then Qwen2.5-VI--2B (Bai
5

---

Published as a conference paper at ICLR 2026
et al. 2025) to automatically generate multiple question-amsyee answer pairs. (3) QA Filtering. Finally, implement a stringent QA filtering process to ensure the quality of the generated QA pairs. The QA filtering stage is to verify that all questions through analysis of the image content. The final QA filtering stage is crucial to prevent information leakage and guarantees that the model must perform true visual understanding, rather than relying on external knowledge or cues within the question itself to answer the generated questions.
strictly aisually-grouddee and answerable exclusively
Specifically, the filtered of QA pairs, denoted as 2, is then defined as:
2 = {lg. a) E D Mv, (q, 1) A Mv, (4) + a).
(4)
where (g, a) is a question-answar pair from the initial generated dataset D, I is the corresponding input image, My, is the LVLM used in QA Filtering, .Mv, (q, represents the answer generated when conditioned on both the question 9 and the image I, and My, (9) is the answer generated when the image is omitted. According to Eq. (4), the QA filtering step ensures that each selected QA pair requires the image context to be answered correctly. To manage computational costs effectively, the QA filtering step is performed using the Q model (Bai et al. 2025 as MM,
After filtering, retain approximately 75k images along with their corresponding QA pairs train the CapRL captioning model. Please refer to Appendix E HaddnHip Hor the curation details.
3.2 CAPRL-5M DATASET
By employing our carefully designed CapRL training scheme, obtained CapRL-3BB, and further used this powerful captioner to annotate 5M images, ultimately forming CapRL-5M.
Image Collection and Processing. In collecting images, primarily considered diversity, quality. and safety, Among the currently high-quality open-source image datasets, ShareGPT4V-IM (Chen) al. 2024b) and DenseFusion-I1 (Li al. 2024 relatively large in scale. Since both datasets have already undergone extensive filtering and clustering to ensure image quality, we directly incorporated all images from them. To further enhance dataset diversity, we also gathered large number of images from the web, spanning natural photographs, documents, charts, and user interfaces. However, the quality of web images is highly uneven, and they pose potential safety risks, which could severely impact both model training and deployment safety. To address this, we applied rigorous filtering and ultimately retained 3M high-quality images. Combined with the two open-source datasets, this yielded total of 5M images. The detailed filtering process is described in Appendix G
Caption Model selection. In typical multimodal pretraining scenarios, the pretraining dataset often requires massive number of image-text pairs, making annotation costs substantial. Considering practical applications, we decide to train a highly lightweight yet powerful captioner to keep annota- tion costs more acceptable. Specifically, we initialize the policy model with Qwen2.5-VL-33 and employ our CapRL framework, resulting in the powerful CapRL-3B model as the captioner.
EXPERIMENTS
4.1 PRETRAINING SETTING
To thoroughly evaluate the quality of the CapRL-5M dataset, we conduct comprehensive comparisons with widely used caption datasets from the spen-source community.
Implementation Details. In our setup, the language model is initialized with a pretrained LLM, the visual encoder with a pretrained ViT. and the MLP projector randomly, following standard multimodal pretraining scheme. We conduct experiments under three settings: Qwen2.5-3B + Qwen2.5-Vit Qwen2.5-7B + Qwen2.5-ViT, and M22.57L CL.P-VVT-LL Training follows the ShareGPT4V paradigm in three stages: Initial Alignment with BLIP-558K dataset (Li al. 2022); Further Pretraining with diverse high-quality image-caption datasets; and SFT with Open-LLaVA- NeXT-IM (Chen & Xing 2024). For comparison, adopt strong baselines including Vanilla, which skips Further Pretraining, ShareGPT4V- IM, DenseFusion- 1M, and CapRL- (randomly sampled from CapRL-5M). Detailed training details provided in Appendix
Main Results. As shown in Table I when using CapRL- IM as the further pretraining dataset, performance on the vast majority of benchmarks surpasses both ShareGPT4V-1M and DenseFusion-

---

Published as a conference paper at ICLR 2026
Table 1: Performance comparison using different pretraining datasets. CapRL-IM significantly outperforms other datasets across all 3 settings, and further improvements are observed when scaling the data to 5M. The best results are bold and the second-best results are underlined.
Pretraining Dataset
Info VQA
Doc VQA
Chart QA
WorldQA
Real
Math Vista
SEED2
Plus
MME
RW
MMStar
MM
GQA
Avernge
Qwen2.5.31
38+
Qwen2.5.Vii
Vanilla
43.9
81.0
72.7
41.6
56.6
30.5
68.6
41,0
61.5
55.5
ShareGPT4V-IM
46.1
82.4
74.2
55.0
44.7
60.5
29.8
68.9
45.2
42.4
70.1
61.4
56.7
DenseFusion- IM
49.4
84.6
74.4
54.1
44.6
59.1
30.7
69.0
45.6
40.2
70.4
62.5
57.1
CapRL- IM
56.2
87.3
78.0
55,1
45.5
62.0
30.3
70.5
47.0
50.0
72.9
61.6
59.7
CapRL-5M
61.5
90.0
80,5
57.6
48.1
63.2
30,9
73.1
50.4
52.6
74.7
62.6
62.0
2we72.55-78
Qwven2.5
V-T
Vanilla
47.6
83.7
77.1
55.9
47.4
60.4
29.4
72.1
48.1
47.1
72.4
62.7
58.7
ShureGPT4V.IM
49.8
85.1
75.7
56.8
46.6
60.9
31.8
71.9
48.4
45.9
72.2
62.7
59.0
DenseFusion- IM
53.5
87.8
76.7
58.6
46.3
61.0
31.1
72.6
48.6
49.7
72.5
63.
60.2
CapRL-IM
59.9
89.5
58.9
50.4
63.1
32.2
72.1
50.5
63.2
62.2
CapRL-5M
63.4
91.4
81,5
61.4
50.8
63.2
34,9
72.7
52.6
52.6
76.9
63.8
63.8
InteinLM2,5578
CLIP-VTTL
Vanilla
37.4
73.2
68.7
56.9
44.2
58.2
30.7
70.7
47.0
43.1
71.8
64.9
55.6
ShoreGPT4V-IM
38.9
73.8
56.3
44.8
33.2
72.6
46.2
43.3
72.7
65.0
56.4
DenscFusion-IM
39.3
76.4
70.8
59.7
44.5
60.3
34.1
72.2
47.9
44.0
73.7
65.5
57.4
CapRL-IM
43.3
80.0
58.0
62.8
34.1
73.4
50.2
46.6
59.6
CMPRL.5M
47.0
83.5
77.7
59.7
50.4
63.5
38,9
73.7
53.3
54.3
77.6
66.3
62.2
Table Ablation on image sources. We annotate the images in ShareGPT4V-IM and DenseFusion- IM using CapRL-3B, and use them respectively pretraining datasets for comparison.
Pretraining
Info
Doc
Chart
Real
Math
SEED2
MMF
MMB MMStar MMVer AID GQA Average
Dataset
VQA
VQA
QA
WorldQA
Vista
Plus
RW
Qwen2.5.30
Qwe.2.5.V
Vanilla
43.9
81.0
72.7
55.1
41.6
36.6
30.5
68.6
44.7
41.0
68.3
61.5
55.5
ShareGPTAV.IM
74.2
29.8
68.9
45.2
70.1
56.7
CApRL-SharGGPTWW-I
52.1
85.9
75.2
56.3
45.6
60.0
30,9
70.9
46.7
47.5
71.4
61.7
58.7
DenseFuusion-I IM
49.4
84.6
74.4
54.1
44.6
59.1
30.7
69.0
45.6
40.2
70.4
62.5
57.1
CapRL-DenseFusioon-Il
55.0
87.8
77.5
56.2
44.7
62.8
32.0
71.0
46.6
49.9
72.7
62.3
59.9
1M. Specifically, under the Qwen2.5-3B + Qwen2.5- ViT setting, it exceeds DenseFusion- by 6.8% on Info VQA, and outperforms by 2.7% and 3.6% on Doc and ChartVQA. These remarkable results indicate that CapRL-3B is effective for domains such as documents, charts, and infographics, which demand fine-grained perception and structured description. The captions in CapRI. are highly detailed and accurate for such image types, enabling LVLMs to achieve better modality alignment and a deeper understanding of the corresponding visual features.
CapRL-5M further demonstrates consistently superior performance across all 12 benchmarks. These results highlight the strong scaling properties of the CapRL-3B ----3---nnnttted dataset: as the training data size expands from IM to 5M, model performance continues to improve steadily. This phenomenon underscores the practical value of CapRL. for multimodal pretraining, as it enables the construction of high-quality, scalable datasets at very low annotation cost.
Ablations about Image Sources. In the previous comparisons, the images used in each dataset are identical. To better control for this variable, we fix the set of images and instead compare the effect of caption quality of different datasets under the Qwen2.5-3B + Qwen2.5-Vis setting. As shown in we compare CapRL with ShareGPT4V- M and DenseFusion- IM. The results demonstrate that, when using the same set of images, further pretraining with the CapRL-3B -nnnotatee dataset enables the LVLM outperform the baselines by more than 2%. This finding indicates that the substantial advantage of the CapRI. dataset over the baselines largely stems from the superior quality of its captions, rather than from differences in image diversity.
Table 2
Scaling Trend Comparison of Different Datasets. We further compare the scaling trend of CapRL and DenseFusiee under Qwen2.5-3B + Qwen2.5-ViT setting. Specifically, we sample different numbers of image-caption pairs from each dataset for pretraining, As shown in Figure 4 the CapRL dataset consistently outperforms the corresponding DenseFusion dataset across various scales of pretraining data. Moreover, the overall trend indicates that this performance gap continues to widen as the data size increases. This phenomenon highlights the strong scaling properties of the CapRL dataset, thanks its high-quality captions, LVLMMs continue to benefit as the dataset size grows
7

---

Published as a conference paper at ICLR 2026
ChartQA
SEED2Plus
IntoV
AI2D
78.0
63.5
58.0
73.0
62.0
55.0
72.0
75.0
74.0
53.0
49.0
70.0
CapRL
57.5
CapRL
46.0
CapRL
69.0
CapRL
DenseFusios
DenseFusion
DensePusion
DessePusko
400k
1M
200k 400k
Pretraining
Preizwining
Pretreining
Pretrnining
Figure 4: The scaling performance comparison between CapRL-IM and DenseFusion- IM. We use different amounts of pretraining data from the two datasets to observe the scaling trend.
Table 3: Captioning ability comparison in Prism Framework. CapRL-3B achieves comparable performance Owen2.5-VL-72B. and significantly surpasses existing strategies that use LVLM-as- a-Judge as the reward. The best results are bold and the second-best results are underlined.
Caption
GRPO Trained
Chart
CharQA
Info VQA
MMMU
Math Verse
Char
Math Vision
Model
Pro
Pro
Xiv
Math
MMStar
SEED
MMMU
Average
Owen2.5-VL-3B
X
65.6
27.1
40.2
28.6
32.8
21.8
54.4
22.6
46.4
64.1
35.1
39.9
Owen2,5-VL-72
X
74.9
35.4
56.4
30.1
36.4
24.8
57.0
23.3
50.7
67.1
37.9
44.9
Qwen2.. V1.
38.0
30.7
39.4
Unified -udgee
/
54.9
25.1
33.6
28.1
34.6
20.4
58.2
24.5
45.4
61.2
36.3
38.4
71.4
34.2
49.3
29.1
33.8
22.9
54.3
24.1
47.7
64.5
36.4
42.5
CapRI-3B
80.5
39.9
64.8
30.7
36.4
32.4
60.1
23.4
55,0
70.6
38.1
48.3
140
Qwend. VL-38
81.5
Relatvve Soore (o
1.00
CayRL DiclletDChutt2
Cap8I-NNmeal-206
28.9
26.5
General Benchmark
Understeninng Renchmark
Accurayy
77.8
77.1
79.9
70.3
120
76.3
24.1
70.6
1:0
50.0
67.7
66.
69.
100
66.5
ClanQA
ChrrQQA
CharAiv
MMStur
SFED
63.1
64.1
40k
SEED
Benchmark
GRPO Training
Figure (Leftt) CapRL demonstrates strong generalization even when trained on images from single domain. CapRL-DoeChart-222 refers to training conducted solely on document or chart images, while CapRI-NNttrrl-200 is trained exclusively on natural images. Both models achieve significant improvements over the baseline out-of-domaan benchmarks, highlighting strong generalization capability. (Right) CapRL demonstrates promising scaling performance on QA training datasets.
4.2 PRISM SETTING
In the previous section, we demonstrated from the pretraining perspective that captions generated by CapRL are highly beneficial for modality alignment. this section, we directly evaluate the informativeness of the captions produced by CapRL-3B through the lens of the Decoupled VQA in Prism Framework (Qiao et al. 2024), and compare our CapRL-3B against other captioning models.
Implementation details. Similar to caption reward design, the Prism Framework decouples VQA into two stages. In Stage 1, the captioner generates captions about the input image. In Stage 2, an LLM answers questions based solely the generated caption. We leverage the Prism framework primarily because it can evaluate caption quality in an objective and stable manner. In our setup, we as the answering LLM, ensuring that benchmark
fix Stage with fine-tuned Dwen2.5-3BB-Intiee performance directly reflects the quality of captions produced by the captioner. To assess the effect of different reward designs in GRPO, (Wang et al., (025). and the other with Qwen2.5-VL. VL-3BB as the judge for caption quality evaluation. The corresponding prompts are provided in Appendix E
include two other baseline models: one trained with
Comparison with Qwen2.5-VL series. As shown in Table CapRL-3B significantly outperforms both the 3B and 7B models of the Qwen2.5-V1 VI series, achieving performance comparable to that of the 72B model. In chart and infographic understanding, CapRL-3B surpasses Qwen2.5-VI. VL-333 by 14.9% 12.8%, and 24.6% on CharlQA, ChartQAPro, and Info respectively. For natural image understanding, it leads Qwen2.5-VL-3B by 9.6% and 6.5% MMStar and SEED. These results demonstrate that GRPO training has substantially unlocked the potential of Qwen2.5-VL-3B. enabling it to fully leverage its inherent knowledge to organize all objects and their attributes within an image into comprehensive and detailed captions. As a result, its perception capability is pushed to the limit, reaching level comparable to that of the 72B model.
8

---

Published as a conference paper at ICLR 2026
Table 4: Analysis of the number of QA per image.
Table 5: Ablations about Sampling Rounds N.
Caption Model
ChannQ Pro
Info VQA
Sampling Rounds
CharQA Pro
Info VQA
MMMU
MMS WeMath Avg
MMMU
MMSiar
WeMath
Avg
Qwea2.5- 5-L
27.1
40.2
35.1
46.4
54.4
40.6
N=1
35.4
58.1
36.5
50.2
56.1
47.3
CapRL-1QA 20k
35.5
36.6
50.8
49.3
56.9
47.6
CapRI-2QA-200 20k
36.8
60.2
37.6
51.1
56.6
48.5
Nu4
36.7
59.9
37.1
50.9
57.3
48.4
CapRL-3QA-20k
36,9
60.3
36.9
51.3
56.8
48.5
36.9
59.6
36.5
50.8
57.7
48.3
Comparison with LVLM-as-a-Judge reward. In our comparison with other reward design methods, observe that when using UnifiedReward JniffedRewarrd222--wween 2.0-qwen-3b as the judge evaluate caption quality, the model's captioning ability actually deteriorates during GRPO training. We attribute this to the during its training, was exposed to large number of captions from text-to-image datasets, which are typically short and only describe the main objects. As a result, the UnifiedReward model tends to favor shorter captions. As shown Fig. T the average caption length during training continuously decreases and eventually collapses to producing only ":description". Conversely, when using Qwen2.5-VL-3B VL--3B as the judge, the bias is in the opposite direction: it prefers overly verbose captions. This makes the policy model prone to exploiting the bias by generating long passages of content irrelevant to the image, thereby satisfying the judge model's preference. As shown in Table the captioning ability under this reward shows significantly inferior to CapRL. Specific examples of such cases are illustrated in Fig. 9 Fig. 10 Fig.
severe bias present in (ifeeddRewar-22.--weee.
4.3 COMPREHENSIVE DISCUSSION ABOUT CAPRL
In this section, we provide a comprehensive analysis and discussion of CapRL. These results further confirm CapRL's general applicability, robustness, and effectiveness.
CapRL demonstrates strong generalization even when trained on images from a single domain. We further investigate the effect of different image sources in the QA dataset used for GRPO training. To this end, we classify the images into two categories using Wwen2.5-VL-3B- (1) documents, charts, or infographics, and (2) natural images. From each category, we sample 20k images for comparison. As illustrated in Fig. 5 (Left), models trained exclusively on chart-type images via GRPO exhibit substantial gains over Qwen2.5-VI.-3B. not only in document and chart understanding but also in general VQA tasks. This demonstrates the strong generalization of CapRL-induced captioning improvements beyond the domains encountered during training.
CapRL demonstrates promising scaling performance on training data. We conduct training on different amounts of QA data to evaluate the scaling behavior. As shown Fig. 3 (ighht, the model's performance improves steadily as the amount of QA data increases. These results indicate that our CapRL framework exhibits highly promising scaling potential. With the continued expansion of the training data, the captioning ability can be further enhanced, unlocking additional potential of Qwen2.5-VL-3B. Given its relatively small parameter size and excellent scaling properties, this approach holds strong promise for application industrial-scale multimodal pretraining.
Sparse QA supervision is sufficient for CapRL. We further examine the effect of varying the number of QA pairs per image. Specifically, we randomly sample 20k images that retain three QA pairs after filtering, obtain 1apRL-3QA-200 after training. By controlling the number of QA pairs per image, we also construct CapRL- -ALLL-QQA-00 IQA-20k and CapRL-2QA 20k.. The results, presented in Table show that even with only a single QA pair per image, D0we2.55-VL-3B achieves substantial improvement in captioning performance, averaging 7.4% higher than the baseline and only 0.5% lower than CapRL-2QA-20L, This highlights the remarkable efficiency of CapRL: highly sparse QA supervision is sufficient to unlock significant gains in captioning ability.
Ablations about sampling rounds N. Results are shown in Table 5] performance improves steadily when increases from to 4, and reaches saturation at = 8. The relatively poor performance at N can be explained by the fact that each question is answered by the LLM only once, without sufficient shuffling of the options. Due to inherent option biases in the LLM, the measured accuracy fails to serve reliable proxy for reward, thereby misdirecting the optimization of the policy model.

---

Published as a conference paper at ICLR 2026
5 CONCLUSION
In this work, we introduce CapRI., a novel framework that successfully applies RLVR to the subjective task of image captioning. By redefining caption quality based on its utility in enabling a vision-free LLM to accurately answer questions, we create robust, objective reward signal. Our results show that CapRL effectively encourages models to generate dense and precise image descriptions, which in turn substantially promote modality alignment in LVLM pretraining. This work marks a significant step away from the restrictive, data-hungry SFT paradigm for RLVR open-ended tasks.
ACKNOWLEDGMENTS
This project is funded in part by Shanghai Artificial Intelligence Laboratory, Shanghai Innovation Institute, the Centre for Perceptual and Interactive Intelligence (CPII) Ltd under the Innovation and Technology Commission (TTC)'s 's InnoHK, Dahua Lin a PI of CPI under the InnoHK.
10

---

Published as a conference paper at ICLR 2026
ETHICS STATEMENT
Our research does not involve human or animal subjects, and all datasets are either publicly available or collected from the web under open-source licenses. To mitigate potential privacy risks, we applied strict filtering procedures and carried out manual checks to ensure that no sensitive, unsafe, or personally identifiable content was included in the training data. The proposed methodology improves model performance without generating additional manual annotation costs. This reduces reliance on data labeling practices that may involve exploitative labor. We believe the benefits of this work, including enhanced accessibility, more accurate multimodal understanding, and stronger research reproducibility, will bring positive impact to society. For instance, it can help visually impaired individuals better understand wide range of images. To further encourage responsible deployment, we provide complete documentation of our methods. Our work complies with relevant ethical standards, legal requirements, and the ICLR Code of Ethics.
REPRODUCIBILITY STATEMENT
We have made significant efforts to ensure the reproducibility of our work. The main paper clearly describes the CapRL framework in Section 3 and the experimental settings in Section 4 Detailed implementation steps, hyperparameters, and training configurations To support dataset reproducibility, we describe the QA curation process Appendix G H and the
provided in Appendix
image collection and filtering pipeline Appendix G which together yield the CapRL-SM dataset. Additional qualitative examples are included in Appendix AA demonstrate model outputs across diverse scenarios. Comprehensive results and ablations are presented in Tables and Figures, enabling transparent comparison with baselines. All proofs of concept are explained with sufficient detail to allow independent replication.
REFERENCES
Amro Abbas, Kushal Tirumala, Dániel Simig, Surya Ganguli, and Ari S Morcos. Semdedup: Data- efficient learning at web-scale through semantic deduplication. arXiv preprint arXiv:2303.09540, 2023.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijic Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint 23X2v:2502.13923, 2025.
Nicolas Carion, Laura Gustafson, Yuan-Ting Hu, Shoubhik Debnath, Ronghang Hu, Didac Suris, Chaitanya Ryali, Kalyan Vasudev Alwala, Haitham Khedr, Andrew Huang, et al. Sam 3: Segment anything with concepts. arXiv preprint arXiv:2511.166 2025.
Soravit Changpinyo, Piyush Sharma, Nan Ding, and Radu Soricut. Conceptual 12m: Pushing web-scale image-text pre-training to recognize long-tail visual concepts. Proceedings of the IEEE/CVF conference on computer vision and pallern recognition, pp. 3558-3568) 2021.
Guiming Hardy Chen, Shunian Chen, Ruifei Zhang, Junying Chen, Xiangbo Wu, Zhiyi Zhang, Zhihong Chen. Jianquan Li, Xiang Wan, and Benyou Wang. Allava: Harnessing spt4v-synthesized data for lite vision-language models. arXiv preprint arXiv:2402.1 1684, 2024a.
Lin Chen and Long Xing, Open- -een-tlava-next: An open-source implementation of llava-next series for fa- cilitating the large multi-modal model community. |nttps://githbb.c com/ xi acachen988 LaaVA-LLaVA-Ne--. 2024.
Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Conghui He, Jiaqi Wang, Feng Zhao, and Dahua Lin. ShareGPT4V: Improving large multi-modal models with better captions. In ECCV, 2024b.
Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc V Le, Sergey Levine, and Yi Ma. SET Memorizes. RL Generalizes: comparative study of foundation model prstnng. arXiv preprint 2rX11:2501.17161, 2025.

---

Published as a conference paper at ICLR 2026
Henghui Ding, Chang Liu, Shuting He, Xudong Jiang, Philip HS Torr, and Song Bai. Mose: A new dataset for video object segmentation in complex scenes. In Proceedings of the IEEE/CVF international conference computer vision, pp- 10224-202340 2023.
Lijie Fan, Dilip Krishnan, Phillip Isola, Dina Katabi, and Yonglong Tian. Improving clip training with language rewrites. Advances in Neural Information Processing Systems, 36:35544-3355 -35755, 2023.
Anisha Gunjal, Anthony Wang, Elaine Lau, Vaskar Nath, Bing Liu, and Sean Hendryx. Rubrics as rewards: Reinforcement learning beyond verifiable domains. arXiv preprint arXiv:2507.17744 2025.
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseck-R1: Incentivizing reasoning capability in Ilms via reinforcement learning, arXiv preprint arXiv:2501.12944 2025.
Alexander Gurung and Mirella Lapata. Learning to reason for long-form story generation. arXiv preprint M.Xiv:25003.28265 2025.
Andrej Karpathy and Li Fei-Fei. Deep visual-semantic alignments for generating image descriptions. In CVPR, 2015.
Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James V Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, et al. Tulu 3: Pushing frontiers in open language model post-training. arXiv preprint 1.. 2024.
Jinsong Li, Xiaoyi Dong, Yuhang Zang, Yuhang Cao, Jiaqi Wang, and Dahua Lin. Beyond fixed: Training-free variable-length denoising for diffusion large language models. arXiv preprint PrXiv:2508.008/1 2025.
Jinsong Li, Xiaoyi Dong, Yuhang Zang, Yuhang Cao, Jiaqi Wang, and Dahua Lin. Visual self-refine: pixel-guided paradigm for accurate chart parsing. arXiv preprint 17X1v:2602.16455 2026.
Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-inan -mage pre- training for unified vision-language understanding and generation. In International Conference Machine Learning, pp. 12888-12900. PMLR, 2022.
Xiaotong Li, Fan Zhang, Haiwen Diao, Yueze Wang, Xinlong Wang, and Ling- Yu Duan. Densefusion-Im: Merging vision experts for comprehensive multimodal perception, arXiv preprint arXiv:2407.08333, 2024.
Chin-Yew Lin. ROUGE: A package for automatic evaluation of summaries. In ACL, 2004.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lec. Visual instruction tuning. In NeurIPS. 2023.
Yuhong Liu, Beichen Zhang, Yuhang Zang, Yuhang Cao, Long Xing, Xiaoyi Dong, Haodong Duan, Dahua Lin, and Jiaqi Wang. Spatial-ssrl: Enhancing spatial understanding via self-supervised reinforcement learning. arXiv preprint arXiv:2510.27666 2025a.
Zijun Liu, Peiyi Wang, Runxin Xu, Shirong Ma. Chong Ruan, Peng Li, Yang Liu, and Yu Wu. Inference-time scaling for generalist reward modeling. arXiv preprint arXiv:2504.00 02495, 2025b.
Ziyu Liu, Zeyi Sun, Yuhang Zang, Xiaoyi Dong, Yuhang Cao, Haodong Duan, Dahua Lin, and Jiaqi Wang. Visual-RFT: Visual reinforcement fine-tuning. In ICCV, 2025c.
Xun Lu. Writing-zero Bridge the gap between non-verifiable problems and verifiable rewards. arXiv preprint arXiv:2506.00103 2025.
Run Luo, Lu Wang, Wanwei He, and Xiaobo Xia. Gui-rl: A generalist rl-style lasiunel action model for gui agents. arXiv preprint 10445 2025.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. BLEU: a method for automatic evaluation of machine translation. In ACI, 2002.
12

---

Published as a conference paper at ICLR 2026
Zhangyang Qi. Zhixiong Zhang. Ye Fang, Jiaqi Wang, and Hengshuang Zhao. Gpt4scene: Understand 3d scenes from videos with vision- lasoualanguagg models. arXiv preprint arXiv:2501001428, 2025a.
Zhangyang Qi, Zhixiong Zhang, Yizhou Yu, Jiaqi Wang, and Hengshuang Zhao. VIn-rl: Vision- language navigation via reinforcement fine-tuning. arXiv preprint arXiv:2506.17222 2025b.
Rui Qian, Shuangrui Ding, Xian Liu, and Dahua Lin. Semantics meets temporal correspondence: Self-supervised object-centric learning in videos. In Proceedings of the IEEF/CVF International Conference on Computer Vision, pp. 16675-16687, 2023.
Rui Qian. Shuangrui Ding, and Dahua Lin. Rethinking image-to- viveo adaptation: An object-centric perspective. In European Conference on Computer Vision, pp. 329-348, Springer, 2024a.
Rui Qian, Xiaoyi Dong. Pan Zhang, Yuhang Zang, Shuangrui Ding, Dahua Lin, and Jiaqi Wang. Streaming long video understanding with large language models. Advances in Neural Information Processing Systems, 37:1 19336-11936) 2024b.
Rui Qian, Shuangrui Ding, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Yuhang Cao, Dahua Lin, and Jiaqi Wang. Dispider: Enabling video llms with active real-time interaction via disentangled perception, decision, and reaction. In Proceedings of the Computer Vision and Puttern Recognition Conference, pp. 14005-24055, 2025.
Yuxuan Qiao, Haodong Duan, Xinyu Fang, Junming Yang. Lin Chen, Songyang Zhang, Jiaqi Wang, Dahua Lin, and Kai Chen. Prism: A framework for decoupling and assessing the capabilities of vlms. In NeurlPS, 2024.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In /CML, 2021.
Noam Rotstein, David Bensaid, Shaked Brody, Roy Ganz, and Ron Kimmel. FuseCap: Leveraging large language models for enriched fused image captions. In WCAV, 2024.
Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. Laion-5b: An open lcage dataset for training next generation image-text models. Advances neural information processing systems, 01225278-52944 2022.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. DeepscekMath: Pushing the limits mathemat- ical reasoning in open language models. arXiv preprint arX1v:2402.03300, 2024.
Yi Su, Dian Yu, Linfeng Song, Juntao Li, Haitao Mi, Zhaopeng Tu, Min Zhang, and Dong Yu. Crossing the reward bridge: Expanding rl with verifiable rewards across diverse domains. arXiv preprint arXiv:2503. 23829, 2025.
Yanpeng Sun, Jing Hao, Ke Zhu, Jiang-Jiang Liu, Yuxiang Zhao, Xiaofan Li, Gang Zhang, Zechao Li, and Jingdong Wang. Descriptive caption enhancement with visual specialists for multimodal perception. arXiv preprint arXiv:2412.144 14233, 2024.
Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi K1. 5: Scaling reinforcement learning with llms. arXiv preprint 2rXiv:2501.11559, 2025.
Bart Thomee, David Shamma, Gerald Friedland, Benjamin Elizalde, Karl Ni, Douglas Poland, Damian Borth, and Li-Jia Li. Yfcc100m: The new data in multimedia research. Communications of the ACM, 59(2))44-73 2016.
Pavan Kumar Anasosalu Vasu. Fartash Faghri, Chun-Liang Li, Cem Koc, Nate True, Albert Antony, Gokula Santhanam, James Gabriel, Peter Grasch, Oncel Tuzel, et al. FastVLM: Efficient vision encoding for vision language models. In CVPR, 2025.
Oriol Vinyals, Alexander Toshev, Samy Bengio, and Dumitru Erhan. Show and tell: A neural image caption generator. In CVPR, 2015.
13

---

Published as a conference paper at ICLR 2026
Yibin Wang. Yuhang Zang, Hao Li, Cheng Jin, and Jiaqi Wang. Unified reward model for multimodal understanding and generation. arXiv preprint 3rXiv:2503.052.60 2025.
Long Xing, Qidong Huang, Xiaoyi Dong, Jiajie Lu, Pan Zhang, Yuhang Zang, Yuhang Cao, Conghui He, Jiaqi Wang, Feng Wu, al. Pyramiddrop: Accelerating your large vision-language models via pyramid visual redundancy reduction. arXiv preprini arXiv:2410. 17247, 2024.
Long Xing, Qidong Huang, Xiaoyi Dong. Pan Zhang, Yuhang Zang, Yuhang Cao, Jinsong Li, Shuangrui Ding, Weiming Zhang, Nenghai Yu, et al. Scalecap: Inference-time scalable image captioning via dual-modality debiasing. arXiv preprint arXiv:2506. 1984:: 2025.
Qiying Yu, Quan Sun, Xiaosong Zhang. Yufeng Cui, Fan Zhang, Yue Cao, Xinlong Wang, and Jingjing Liu. Capsfusion: Rethinking image-text data at scale. In CVPR, 2024.
Tianyu Yu, Bo Ji, Shouli Wang, Shu Yao, Zefan Wang, Ganqu Cui, Lifan Yuan, Ning Ding, Yuan Yao, Zhiyuan Liu, et al. RLPR: Extrapolating rlvr general domains without verifiers. arXiv preprint arXiv:2506. 18254, 2025.
Pan Zhang, Xiaoyi Dong, Bin Wang, Yuhang Cao, Chao Xu, Linke Ouyang, Zhiyuan Zhao, Haodong Duan, Songyang Zhang, Shuangrui Ding, et interilmxcooposeer A lisiun-language large model for advanced text-image comprehension and composition. arXiv preprint arXhv:2309.1511. 2023.
Zhixiong Zhang, Shuangrui Ding, Xiaoyi Dong, Songxin He, Jianfan Lin, Junsong Tang, Yuhang Zang, Yuhang Cao, Dahua Lin, and Jiaqi Wang. Sec: Advancing complex video object segmenta- tion via progressive concept construction. arXiv preprint 2507v:250..15852 2025.
14

---

Published as a conference paper at ICLR 2026
A CAPRL CASES
We provide further illustrative examples of CapRL-3B to highlight surprising captioning capabili- ties in this section.
Comparison with Qwen2.5-VL-3B. As illustrated in Fig. 6 CapRL-3B demonstrates remarkable capability in understanding infographics, providing information that is both comprchensive and accurate. In contrast, Qwen2.5-VL, ----3B,, as shown Fig. makes numerous errors in identifying key information within infographics. Furthermore, Fig. highhights that CapRL-3B achieves substantially higher accuracy in chart understanding compared to Qwen2.5-VL-33. Similarly, in the case of physical image understanding in Fig. 12 CapRL-3B also demonstrates clear superiority.
Comparison with Unffie/WW-as-Jugee-- the issues introduced by L.V..Mas--- rewards, we present examples of captions produced by models trained with different methods in Fig. [ Fig. 11 and [0] The
and Qwen2.5VI--s--Judeee
To intuitively illustrate
model tends ignore key visual information the image and outputs lengthy, irrelevant content, such repeatedly asserting that its caption is of high quality in order to exploit reward hacking. In contrast, UnifiedR--as-Juune -uddeee produces overly short captions that omit substantial amounts of critical chart information.
More cases CapRI.-3B in understanding infographies and natural images. Fig. 13 and Fig. provide additional evidence of the impressive perceptual capacity demonstrated by CapRI-33B.
B MORE ANALYSIS EXPERIMENTS ABOUT CAPRL
Leaking QA Data Leads to Performance Degradation. We randomly sample 20k in- stances and construct two training conditions: one using the retained QA and the other us- ing the filtered QA. As shown in Table 6, the model trained on the leaking data performs on average 1.1% worse than the one trained on high-quality data. This indicates that leak- ing QA introduces spurious reward signals that mislead the optimization of the policy model. Even when captions are not closely aligned with the image content, the LLM may still achieve high answer accuracy, thereby preventing higher rewards from being correctly assigned to genuinely better captions.
Table 6: Comparison between training with data con- taining leakage issues and training with filtered data. Leaking data leads to an obvious performance drop.
Training Data
ChariQA Pio
MMMU MMStr WeMath Avg
VQA
20aking
36.4
58.9
36.1
50.7
55.1
47.4
Retined20k
36.8
60.2
37.6
51.1
56.6
48.5
C COMPARISON WITH GRPO-TRAINES VQA MODEL.
We incorporated Format Reward during the RL training process the 20k dataset, ensuring the output adheres to the format: After training with this format, obtained Reasoning-3B--25 As shown in the result:7 while Bassoning-3B-220 achieves a noticeable improvement over 5 Owen2.5-VLL-3.. it falls significantly short of CapRL-3B. Specifically, CapRL-3B surpasses Reasoning-3B- across all 11 benchmarks. This substantial margin highlights the superiority of the CapRL algorithm, demonstrating that our method effectively activates the model's I's captioning capabilities through sparse QA supervision applied directly to the caption.
by average of 3.8%
Table 7: Comparison with GRPO-trained VQA model.
Caption Model
ChartQA Pro
Info VQA
MMMU
MMStar
WeMath
Average
Qwen2.5. V-LBBB
27.1
40.2
35.1
46.4
54.4
40.6
Reasoning-3BB -BB0
32.7
46.2
36.0
47.2
56.1
43.6
CapR1-3B-22kk
36.9
59.6
36.5
50.8
57.7
48.3
15

---

Published as a conference paper at ICLR 2026
CapRL-3B:
The image is an infographic titled "Kred 2012: 10 Influential Team
Olympians," highlighting from Team USA in 2012, particularly focusing on their Olympic
influential Olympic athletes
Kr
Kred 2012
Influential Team USA Clympans
medals and social media influence scores. Here is detailed description:
999
Header and Awards:**
The image features three main athlctes with their respective modals
and Kred scores:
LeBron James (Gold Modal in Basketball) with Kred score of 999 and medal.
977
Michael Phelps (Silver Medal in Swimming) with a Kred score of 977 and 6 medal.
Gold
Hope
Medal Soccer)
977 and
a medal.
Silver
2. *Stuirruse of Medals:
Bronze Hoco
The image uses a graphical representation with blue background udorned with stars, indicative an American Lehron James with the highest Kred score, followed by Michael Phelps and Hope with identical scores.
top shows
CRAMP
3. *Social Medin Metries:
Followers for cach athlcte:
Michael Phelpe 689, 591 followers
-LeBron James: 5,627,723 followers (highest)
Hope Sole: 542.813 followers
992
975
937
B12
B9S
BB1
BBO
Retweeted times and replics received:
PPbclpp: retweets. 75,695 replics
Jammes: 1,072,729 retweets, 1,250.351 replies
Waack ev
adteeiie cabilrtly aberdarr
Soolo: 75,014 setweets, 148,310 replics
*AAtllttes List with Kred Scores Different Events:
section ten athlctes along their
Kred scores:
and points Lochte (Swimming) with 975 and points
Durant (Raskethall)
Wieber (Gymmastios) with 937 and 5 points
Raisman (Gymmastics) with 912 and points
K
How Koad icores caleulaee
Douglas (Gymmastics) with 899 and points
US0 per comeer oalen poneerr sryiinn conetrme
Franklin (Swimming) with 891 and 5 points
RRiharrds-Ror
points
**Celebrity
A scetion highlights which events winning celebrity atsention:
Wendy Williams (g Wendy Williams) on 30 July: Talking about "GIRL POWER, GO TEAMUSA. #LONDONOL with link
usagym.org pages/post.htm
Samantha Giancola (iMMTVsammii on 30 July: Loving swin swimming, Soccer track, and gy symaastics the best
Mergan July: Commeuting Michacl Phelps being "official Greatest medal-winner in Olympic history." pointing @MMicheePheelps
Bill Simmons (3 Julycmentioned Swedish-Norway women's handball coming up minutes, noting
plenty of seats available on the Swedish bandwagon
** **Keed Score Calculation:
The image explains that Kred scores are calculated based on influence, which is the ability to inspire action through social media, seored on helping spread messages through social media.
1,000-point Outreach
by generosity in engaging
Lago Credits: logo bottom infographic to Neez, dualindex hottom.
Figure An illustrative example of CapRL-3B applied to infographic understanding.
D LLM ABLATION IN CAPRL
We employed the Qwen2.5 series as the answer-model, utilizing five different model sizes ranging from 0.5B 32B to answer questions based on the captions. We observed in that even when using the smallest 0.5B model, CapRL still yields significant improvement in the captioning capability of Qwen2.5-VL VL-3BB however, there noticeable performance drop compared to the 3B model. Performance improves marginally from 1.5B to 3B and then begins to saturate, with significant variance observed between the 7B and 32B models. These findings indicate that the answer-model does not require excessive parameter size; it simply requires fundamental capability for information extraction. However, extremely small models, such as the 0.5B variant, appear to lack sufficient capacity for effective extraction.
16

---

Published as a conference paper at ICLR 2026
Qwen2.5-VL-33
This image is graphical representation showeasing the most influential Team USA Olympians in 2012, as Twitter's "Kred" platform. manking based both influence and outreach metrics, The image predominantly features athletes from the USA Olympic teum, inc huldning LeBron James, Hope Solo, Michael Phelps, and
Kr
Kred 2012
10 Influential Toaam USA Olympians
999
977
977
At the top of the image, there are three main athletes highlighted with star ralings und scores:
Gold LAAATING
Fred the Teacher* Has
and rated
a "Golden Medal."
2. **Mary Smith** Has score of 977 and is rated a
Silver
*Chriistpheer Brown**; Has 977 and
Mishco Fhoto
Bronze
raled a "Subile Medal."
albletes, there
4. #Funniic Lawrence** Has of 977 and rated a "Yellow Medal," representing swimming.
score of 992 and is rated a "Gold Modal," representing basketball.
*Honny CEO**
oee sirees
aggeralttioo effendraeloo Wha ofrice
Additionally, there is a section titled "Which events and athletes are winning celebrity atteution?" where:
(*) Fisheries**; Talks about the team's performance
975
937
912
699
891
BBO
and the Olympics.
**) fitting Howard** An article by Howard, discussing the Swodish-Norvegian handball match.
Wiith
aherlocc
+* Matcching** Discusses the Mexico-US US Handhall
match.
POUC Co aoarato
Seroe tick anduund mundly
500
explaining Kred calculated." " which details that influence and outreach factors in the scoring system, defining it as "the ability to inspire action through social media."
dovoit harball vald
Genteet
Hallucinations!
Kr
How Krned scores are cafcelated
Interee ebly replocddio wold sooe lejing tee scrent her menegg nough sndil mock.
Figure 7: An illustrative example of Qwen2.5-VIL-3B applied to infographic understanding.
Table 8: LLM ablation in CapRL.
ChartQA Pro
Caption Model
InfoVQA
MMMU
MMStar
WeMath
Average
Qwen2.5-VL-3B
27.1
40.2
35.1
46.4
54.4
40.6
Answer0.5b
33.7
54.9
35.8
48.6
56.1
45.8
Answer1.5b
36.2
59.5
36.7
50.7
56.6
47.9
Answer3b
36.9
59.6
36.5
50.8
57.7
48.3
Answer7b
35.7
60.2
36.9
51.0
56.9
48.1
Answer32b
35.9
60.1
35.7
50.2
56.3
47.6
PROMPT USED
We provide all the prompts employed our experiments in this section. Specifically, the prompt used in CapRL for guiding the LLM to answer questions conditioned captions is illustrated in Figure 15] the prompt for utilizing the Unified Reward Model as the reward model shown in Figure 16 and the prompt for adopting Qwen2.5-VL.B the reward model is presented in Figure 17
17

---

Published as a conference paper at ICLR 2026
CupRL-3B:
The image is a bar churt that presents survey results regurding opinions allowing more, refugres into various countries, specifically Jordan, Lebunon, and Turkey. The survey was conducted by the Pew Research of Global Attitudes Survey, focusing on question Q61. Here is detailed description:
Jordan, Lebanon, Turkey want fewer Syrian refugees allowed into their country
same, fewer Syrian
your optnioon should use alluw more refuugess from..sff into our country. fewer rifugeess about the
now?
Center in Spring 2017
Jorden
8%
8%
90%
23%
"Tiil:: Jordan, Lebanon. Turkey want Rower Syrian refugoos allowed into their country**
Syrien
Jardanian
63
27
"*Questio:=** In your opinion, should we allow more refugees from Syria our country, refugees the do now?
Lebenon
13
The chart shows perceninges for each country and religious othnic group regarding theirpreferred number of refugees:
Sunni
12
46
36
1. **Jordan=**
-8% want "More"
Turkey
38% "About the same"
600% want "Tewer"
Source: Spring Globel Burvey Q62.
"None (VCL)"
**Syrian;** -64% want "More"
-11% want "About the same"
Qwen2.5-VL-3BB
266% "Fower"
(This categorry includes Syrian respondents overall)
Here's summary of the data for each country;
**Jordanian"**
1. x "Jrdan"* More: 8%
33% want "More"
56% "Ahout the same"
About the same: 23% Fewer: 68%
- -635 want "Fewer"
"None (VCL)"
2. **Suddi Arabia* (assumed to be the "Syrian" bar):
**Lebanon.**
-4% want "More"
Abbutt the same: 11% Feewer: 25%
-13% want "About the same"
Hallucinations!
"Fower"
- 44%% want "None (VCL)"
3. **Lebanon
*Shhia.**
About the same: 35%
44% wan: "More"
117% "About the same"
444% want "Fewer"
4. **Hijazi Shia"* (assumed as scparate):
"None (VCL.)
Morr: 46%
About the same: 40%
**Sunni:**
Fewer:
-4% "More"
-12% want "About the same"
"Amrricn ** (assumedd separate):
More: 31%
36% want "None (VCL)*
same:
-Fewer: 31%
**Christian:**
33% want "More"
**Sunni Republic**; -oore: 20%
100% "About the same"
333% want "Fewer"
-33% want "None (VCL)"
Fewerr 36%
**Turkey:**
7. **Christian Fiji"*; More: 33%
44% "More"
99% want "About the same"
About the same: 38% Fewer: 31%
53% want "None (VCL)"
8. *Tuukey** More: 30%
** *ources;**
Spring 2017 Global Attitudes Survey
About the same: 31% Fewer: 39%
Question Q61
Figure 8: Chart understanding comparison between CapRL-3B and Qwen2.5-VL-3B.
F LARGE LANGUAGE MODEL USAGE
In the process preparing this paper, we made use of large language models (LLMs) primarily as a writing assistant rather than methodological component. Specifically, L.LMs employed to
18

---

Published as a conference paper at ICLR 2026
CapRI-33B:
Older Republicans especially likely to see Trump as fighting for their beliefs
The image is a bar chart from the Pew Reseurch Center that illustrates how different groups Republicans and Republican-leanme
individuals view President Trump,
specifically focusing on how they see him as "lighting for what I believe in." The chart is broken down by several categuries, including age groups, educational levels, polirical ideology,
Among Republicans and Republican leaners, who say the phrase fights what betieve describes Trump
respondents full Republicans or leaners towards the Republican Party, Here a detailed description:
Very
Fairly
All Rep/Lean Rep
51
36
87
#** Title:
Republicans especially likely to Trump as fighting for their belieff' nAmong Republicans and Republican leaners, who say the phrase lights for what believe describes Trump..."
Ages 18:29
31
45
76
30-49
41
42
82
50-64
58
92
85+
68
26
94
Key Findings:
**Overall Pereentage (All RepyLeen Rep).**
Postgrad
42
38
80
Very well: 51%
College grad
40
Fairly well: 36%
Some college
51
36
87
-NET (Total pereentage):
HS or less
56
33
89
**Age Groups:*
Ages 18-29: 31% well, 45% fairly well. total 76%
Conserv
63
31
94
30-49: 41% very well, 42% fairly well. total 82%;
-50-64: 58% very well, 33% fairly well, total 92%
Mod/Lib
65+;; 68% very well. 26% fairly well, total 94%
Republican
61
32
93
Older groups (65 and above) are the most likely to see Trump as actively fighting for their beliefs, with 65- age group having the highest pereentage 94%,
Republlican
41
77
Note: Hased on Republl cans and Republican-leanir independents. Souroe: Survey U.S. adults conducted Feb. 4-15, 2020.
** Edducation
Postgrad: 42% very well, 38% fairly well, total 80%
RESEARCH CENTER
College grad: 45% very well, 40% fairly well. total 85%
Some college: 51% very well, 36% fairly well, total
87%
HS 56% very well,
well, total
Individunls with high school diploma or less education are most likely to see Trump fighting for their beliefs (89%), while those with postgrad dogrees are least likely (80%).
*Poliitical Conservatism.
Conserv (Conservative): 63% very well, 3 31% fairly well, total 94%
Mod/Lib (Moderate or Libcral); 32% very well, 44% fairly well, total 75
Conservatives are most likely to see Trump fighting for beliefi,, with a net percentuge of 94%,
5. "Partisanship:*
Republican: very well, 32% fairly well, 93%
Lean Republica:: 36% well, 41% fairly well, total 77
Leaners towards the Republicun Party less likely Trump this way, with net percentage 77%.
*Noots::
The survey was conducted February 4-15, 2020, based on Republicans and Republican-lcaiing independents. Center source data.
Percentages Breakdown by Categories:
-The highest net pereentage (94%) seen the 65++ group andeesssrvatives.
The lowest net pereentage is in 18-29 age group 76%.
Full Republicans (93%) and those leuning Republican (77%) moderately high.
Figure 9: Example of CapRL-3B chart understanding.
refine the readability and fluency of the manuscript by polishing grammar, improving phrasing, and enhancing stylistic consistency across sections. This usage allowed us present the technical content with greater clarity and precision, ensuring that the intended contributions could be more effectively communicated to the reader. Moreover, the models were used to check for potential linguistic errors or awkward constructions that might distract from the scientific substance of the work. Importantly, we emphasize that the role of LLMs in this study was strictly limited to text editing and proofreading. They were not involved in designing the research methodology, formulating experimental procedures, analyzing results, or drawing conclusions. All conceptual development, data processing, modeling. and evaluation were carried out independently by the authors. By constraining the role of LLMMs to
19

---

Published as a conference paper at ICLR 2026
Older Republicans especially likely to see Trump as fighting for their bellefs
UntfedBeewarrlMoodda-----------
Aronn Republicans and Republiom leaners, sny the phrose yfigtt Trump-
This image appears to be a bar chart from a study conducted by the Pew Research Center. The title "Older Republicans espocially likely to see Trump as fighting for their beliefs" is prominently displayed top. The chart compares percentages among Republicans and
in' descrilus
Fairty
All RepyLLan Rep
B1
36
87
Republican leaners on how well the phrase 'ighhss for what believe in' deseribes Donald Trump. across different categories. Key data points are listed with "Very well. " "Fairly well," and "NET" columns for each category. Specific numbers are provided for age groups (e.gg All Rep/Lcum Rep: 51 for "Very well', 36 for FFirly well"). education levels (postgrad, college grad, etc.), political orientation (conservative, moderate/libenn affiliation (Republican, Lean Republican). There numerical values and percentuges (e.g, 92, 85) indicating results for each category. The source and survey details (U.S, adults, Feb. 4-15, 2020) are noted at the bottom. The description captures the structured, comparative data format and the specifics of the study's findings in detail.
Ages 18-29
31
45
76
30-49
41
42
82
50-64
33
92
Postgred
and Republicun
Collegee
40
Some collage
51
36
87
H5 or less
56
33
89
Donserv
94
Mod/LLb
Republiccn
32
Image-related key information is incomplete and inaccurate.
Note: Based Republi plpublicann and Republlcca leening independents. Source: Survey of U.S. adults conducted Fob. 15, 2020.
PPW RESEARCH CENTER
Figure 10: Example of Qwen2.5VV..as.Jdge-3 chart understanding.
language refinement, we aimed to benefit from their strengths in natural language processing while maintaining full intellectual control over the scientific and technical aspects of this research.
G DATA PROCESSING
To ensure both quality and safety, our data processing pipeline consists of three main stages. First, inspired by SemDeDup (Abbas et al. 2023). we construct clusters to identify and remove images with redundant semanties. During this step, we also discard low-resolution and overly simple images, while filtering out content that involves violence, pornography, or other safety concerns. Second, to avoid benchmark leakage, we integrate the images used commonly referenced evaluation datasets and form clusters with them. Any images from our collection that are overly similar to benchmark samples are eliminated. Third, we conduct safety inspection through human verification. Annotators perform sample-based screening, and once the proportion of unsafe images falls to a negligible level, we stop filtering. Following this process, we obtain the final dataset, CapRL-5M.
H QA PROCESSING
In constructing the QA pairs, we employ the Qwen2.5-VL-72) model with prompts shown in Figure 18 For each image, we generate five questions and retain those without leakage issues. We do not de deliberately control the number of QA pairs per image, prioritizing instead the overall dataset size and diversity. As revealed in later ablation studies 4 although even a single QA per image proves highly effective, adding more QA pairs still brings marginal improvements.
During QA filtering, since the model's answers carry uncertainty due to temperature parameter, we do not filter questions solely based on the correctness of a single response. Instead, we sample responses four times for each question, shuffling the answer options each time, and then measure the accuracy of the LVLM's answers both based on the image and based only on the question itself. We ultimately apply threshold to filter out questions with high basee accuracy but low question-only accuracy.
It is worth noting that, because filtering criteria quite strict, some discarded QA pairs contain only mild or even negligible leakage. This also explains why, as shown in Figure training with the leaked data does not cause training collapse but merely led to degraded performance.
20

---

Published as a conference paper at ICLR 2026
I PRETRAINING DETAILS
Model Architecture. In our experimental setup, the language model component is initialized with pretrained LLM, the visual encoder is initialized with a pretrained ViT, and the MLP projector is randomly initialized. This setup corresponds to a commonly adopted starting point in multimodal pretraining. To ensure the robustness of our conclusions, we evaluate three groups of architectures: (1) Qwen2.5-3B + Qwen2.5- ViT, (2) Qwen2.5-7B + Qwen2.5-Vin and (3) InternLM2.5-77 + CLIP- ViT-L. This selection jointly considers differences parameter scale, LLM backbone, and visual encoder type.
Training Setting. Following the training paradigm of ShareGPT4V, our training process consists of three stages: Initial Alignment Further Pretraining SFT. (1) In the Initial Alignment stage, we unfreeze the MLP and perform preliminary alignment using the BLIP-558K dataset. We adopt learning rate of le-3 and batch size of 256. (2) In the Further Pretraining stage, we unfreeze all parameters including the LLM, MLP, and VIT. This stage facilitates further alignment with various high-quality
datasets, enabling the LLM to better understand visual features. We set the learning rate to 4e-5 and the batch size to 256. (3) In the SFT stage, we again unfreeze all parameters and train the OpenL.LaVA-Nens dataset. We set the learning rate to 2e-5 and the batch size to 128.
Baselines. We selected several strong baselines for comparison. (1) Vanilla, which skips the Further Pretraining stage and only goes through the first and third stages, Additionally, we constructed two more baselines by varying the dataset used in the Further Pretraining stage: (2) ShareGPT4V-IM, and (3) DenseFusion- 1M. To ensure a fair comparison by controlling the number of samples, we randomly sampled I million image-caption pairs from the SM dataset to form CapRL-IM.
J CURRENT LANDSCAPE OF MULTIMODAL MODELS
Multimodal learning has accelerated rapidly in the past two years, driven by systems that couple
language with perception and action at scale (Liu et al., 2023] Bai et al. 2025 Li et al. 2025
2026 Xing et al.) 2025 2024 Liu et al., 2025a Carion et al. 2025 Qian et al. 2025; 2024b Ding
ct al. 2023 Qian et al. 2023 These advances Zhang et al. 2023 2025 Qi et al. 2225a b Qian
ct al. 2024a) have produced broad empirical breakthroughs across benchmarks and real-world tasks.
Compared to earlier eras that focused mainly images paired with text, today's 's models are larger, trained on more diverse corpora, and natively support additional modalities such as video and audio.
21

---

Published as a conference paper at ICLR 2026
Qwen2.5VI La-nn.55V-----udeee--
Older Republicans especially likeey to see Trump as fighting for their bellefs
This image is detailed bar graph from a survey conducted by the Pew Republicans and Republicun- leaning individuals view Donald Trump's political leadership using specitic phruse, "fights for what I believe in." Here's a comprehensive description and analysis:
Research Center, which examines
Anony Regrufficurss Repuufiions lennerra, who any the planase fighte for schat befiee in descrbhee Trrspp...
Very well
Fairly
NET
All Reg/Leon Rop
M
36
87
TTTiti and Subject**: The study asks Republicans and Republican- individuals view Donald Trump, focusing on how they see him "fightigg for their beliefs.
18-29
66
30-46
42
82
50-61
08
33
62
*Survvyy Data""; The data is divided into several categories and metric variables, including e' political ideologies, and pun
Prodggee
80
Image-related key information is incomplete and inaccurate.
Col lggg grod
40
85
**Key Observations**:
HS or ross
58
89
AAmong All Repel cun Reps, 87% see Trump as very well or fairly
well deseribed someone who "fights for what believe in."
Danserv
85
31
YYounger age groups (18-29 and 30-49) have lower percentages
Mod/Uh
(76% ages 18-29 and 82% ages 30-49) compared to older groups (51% for agos 50-64 and 68% for ages 65+).
Repull Rap
98
Lean Republican
Education level: Postgrads see Trump this way 80% the time, while those with some college education see him 87% of the time.
hite Boted on Ropu ilkers and Repullikan Heoning indeper dents.
Source Survey of acuta condudedd rub. 4-25, 2000.
Political idcology: Conservatve individuals see Trump this way
94% the time, with moderate liberal (32-44% of the time).
4. Age and Hubri."
Younger politically-awwar generations (e.g. 18-29, 30-49) show ussociation with Trump's leadership. indicating a generational shift in perception.
Older generations (50-64 and 65+) associate Trump more strongly with this phrase, suggesting a deeper understunding of motives.
5. *Educctional Background and Political Affiliatio**:
Higher levels of education correlate with stronger perception of Trump's commitment his beliefs. Conservative individuals overwhelmingly see Trump as fightine for their beliefs, which contractctt or signifies a significant partion of their
loyalty.
Image-unrelated information verbose and lengthy
**Party *ffiliatios**
Repuhlicars and Republican-leuning individuals see I rump as highting for their heliets 9.3% of the time, with leaning Repuhlicans (36% very
well, 41% fairly well) joining this pereeption essentially.
7. **Percentages and Interpretation
highest pereentage (94%) secing Trump light belongs to conservative respondents, indicating their strong belief'nn his idcological commitment.
lowest percentage (36%) among moderate liberal Republicans, signaling that these individuals feel connected to Trump's political ethos compared to the conservative and leaning Republican groups.
8. *Survey Context**
The survey was conducted in l'ebruury 2020, reflecting current political sentiment and personal belief perceptions about Trump's lendership style
9. **Possible Implications**
Thii indicates changes in age-relates political denominations within the Republican party, with older generations having more consistent and stendfast in Trump's dedication his principles.
Education appears to serve cnacial factor in shaping how individuals perceive Trump's political commitment.
There is a strong correlation between party affiliation and belsef in Trump's 's idcological fervor.
**Cooclusion** The data highlights the diverse perceptions of how Docald Trump represents his targets' beliefs, showing variations by age. education, and political alignment. It also underscores the strong ideological and short-term political connection among conservatives, while revealing generutional and educational shifts impacting views on Trump's leadenship style.
This detailed description includes all aspoets of the image, providing a compechensive understanding of the data and its implications. It covers various key demographic catogories, their perceptions, and the overall narrative they present, enabling a pure text model to answer a wide range of questions related this graph effectively. The explanation is structured logically, covering different dimensions of the data, from the percentages observed to the implications these observations, thereby offering thorough analysis of the given information. The description also includes relevant inforences and conclusions, which derived fiom the data presented. The of tett blocks and structured format aids clarity and case of compreheusion, making it easy for a text model to understand and effeetively generats answers to queries based on this description. inclusion of sucietal and political context enriches understunding subject matter. The his camprehensive approach ensures that the description is both informative and engaging, covering all necessory details and providing well-rounded understanding of the graph.
image, providing a broader perspective
This level of detail also enables other Al systems to process and analyze the information accurately. ensuring that they can answer questions
such What age groups see Trump strongly? How does educational background influence perception? What the political affinity link?
These questions can be answered based on the coherent and detailed description provided. The description based on all visible information in image, ensuring
content overlooked or misundasteod. The structuned logical flow makes casy follow, each segment contributing to a comprehensive understanding of the graph and its implications. The description is both factual and insightfill, providing robust basis for any further inquirics or peestice-asswering-reeee tasks. Thus, serves an excellent example how to approach and describe image with precisson and thoroughness.
part
Figure 11: Example of Qwen2.5VLL-asJJudee-BBB chart understanding.
22

---

Published as a conference paper at ICLR 2026
Fewer Hallucinations and More Details
Original Qwen2 SVL-3B:
After
D"Dmmniiionss and Labele
The image depicts mechunical or ongineering schematic of simple mechanical system. The diagam shows a solid rectangular black that is suttonary. The block has dimensions noted millimeten:: witth 200 mm and hright of 400 block
block hegght of 200 indictied
libeled 2200 maa" pointing downoords
block widhh of mm as indicated by biholed 400 mas' nimning horbonntlly from left to right the of the block.
resting
surface
To the right of black, there is single horizomal force Isbolal (PP "eeti KN) (kilonowises) This forze is appliod the center of the block borsentally, indicating horizontal tensile pointing
SSapp Color:
likely represeating concrete metall stab, possilly a govementt or foumértion.
coloeed marcon (magenta)
load depending
right
fiees block puntlicl ficcs.
4. FFrree Applicd:
1 isllonewtoss) upplied to This force shown smow
There are specific dimensions marked, indicating:
right side
The feeer {P 2 'sest KN) is appliol between points that
"P 2.aN" pointing to the right.
The force applied at the top rightt comner of the block.
foundation
postiioned 400 from the left and 200 from the block).
anvirommen to upport weight of block firoz.
Haliscimations!
Figure 12: Example of CapRL enhancing the captioning ability of Qwen2.5-VLL-B.
23

---

Published as a conference paper at ICLR 2026
CapRL-3B:
Social Media Statistics
The image an infographic titled "Social Media Statitiie: Breaking Down Major Social Networks," specifically focusing on Facehook, Twitter, and Pinterost. Below detailed description:
FROACIN DORK IHC MUDD SOOML NETWORS
facebook
77%
SINALLIN Wore ORIY CAR
43%
20 FE
** "oocial Media Statistic**
BLUH FACEION PMI
201 SOCIA CO ORART HA25
"BREAKING DOWN THE MAJOR SOCIAL NETWORKS
Facehook
twitter
**77%** B2C companics have acquired customers through Facehook.
S1G4LAMEE SrOWN AE-RIA 201
56%
**43%*** B2B compunics have acquired customers through Facebook.
Facebook accounts for 1/2 all returgeted clicks the web.
COUOEEL SeNve ortre senrans (00S 190 TWITTER
189 million "mobile only" users and 488 million mobile users.
login more per day.
47% of users say Facebook influences purchases.
REGAACI waw.aiiSiieee T8E IUGEEST co 20M12 81T9611 AWW.ARR ESS
FAE IGHORED
of social users prefer connecting brands on Facebook. Facebook is made of 43% male and 57% female users.
Pinterest
*** Twitter
80%
SPLB ALl
55-64 largest growing demographic, up 76% since 2013.
millioo daily usens, producing million tweets per day.
P0G LITILLSTE FBB15 SCTUU ABR TIBST IOET
69% of all reported follows are suggested by friends.
USESS AAGTEMALE
34% of web marketers have generated leads using Twitter.
32% of weh
using Twitter.
specialtas boand and market enedio platforms. stact Sockyy a
56% of all customer tweets to brands are ignored. Rescarch shows the comes between 4:30 and 5:30 PM green icon with 56% highlights this statistic.
artas.
highost CTR (Click-lhrough
rb.com/queenc tymedia
pinterest
*itt Pinterest
registered users are female (spocifically highlighted icon).
80% of all are "re-pins."
Pinserest referrals spend 70% more than non non-social traffic. get 36% more likes than those without.
Pins with listed
With 17M11+ + engagements, only 15% are on brand-speoific boards.
have purchased
purchase Pinterest.
Additional
Key points about Pinterest:
80% pins are "re-pins."
Pinterest referrals highly effective.
Pins with prices perform better, getting 36% more likes.
Only 15% of engagements
brand-specific boards.
69% of users have parchased or want to purchase via Pinterest,
WHM Footer Information:
Queen City Media specializes
content marketing on major social platforms. today
media evaluation.
Contact details:
www.qemmy.ccm
info@@qemmy.comm com
Phone: 716-845-6500
AAdrress: 515 Main St, Buffalo, NY 14203
Social media links provided:
fb cummpeececit
rwitter centter.com/qqmme com/qemny
pinterest.com/geeen comigemnn.....
Figure 13: An illustrative example of CapRL applied to infographic understanding.
24

---

Published as a conference paper at ICLR 2026
CapRI-33B:
The image depicts screme ourdoor scene leaturing two individuals, likely a couple, sitting on a wooden bench in what appears be garden courtyard decorated in patriotic themes, probably for historical commemorative event given context. Here detailed description:
**Setting and Location:** **
The image is taken in gurden or patio area,
evidenced by the presence of plants, flowers, and garden bench. The setting is adorned with British flags, specifically Union lacks, suggesting, context, possibly location in the United Kingdom.
2. **Subjjcts:**
The image shows man woman sitting side side wooden bench.
Thee woman on left, wearing green dress vintage style, likely 1940s or 1950s, given its emnraald green full skirt. She has blonde hair and wearing sunglasses and dark brown shocs. Sbe holding the man's hand.
*AAttre:**
The woman is dressed in green dress, which is greenish-blue in color, typical of retro or wartime em outfir, possibly from the 1940s.
4. *Mann***
a military uniform, specifically cup (possibly anilor's cup) dark military anchor pins, indicating might be naval or force officer. He wearing cap with badge and a jacket with buttons, typical of military attire from that period.
man right,
5. *Furiiture and Background." we
TTeyy sitting weoden bench that outdoors, likely garden park, indicated by various potted plants and around it. The bench outdoors, possibly on a sidewalk or patio.
6. **lag::**
A large British flag (Union Jack) is prominently displayed behind couple, draped over the back beach. The flag is red, white, inseriptions suggesting old vintage, possibly from ship military context. This flag
and blue, with remnanns spans across the image from left to right.
7. **llag Display: Union Jack.**
TThe flag lurge appears old, with red, white, blue, typical British national flag, of pink middic, indicating might be from historical or naval setting.
8. *Plants and Decor:** 44
The garden is lush with various plants and flowers, including a pot on the left containing greenery in blue pot, likely herbs or small tall green onions garlic chives (likely Allium fistulosum).
plants. There also planted
*Groundd Additional Elements
The ground paved with brick conerete sidewalk, and there a small drain or cover the right side of the image. The bettom left comerr mentions "Mereury Press Media Lid." suggesting the image is from media source.
10. **Text and Setrings:**
sky bright, indicating sunny day, likely spring or summer given well maintained gurden the couple's vintage clothing
11. **
The wearing dark navy blue pants and shoes typical of military uniformity, complete with bell and decorations typical of naval officer.
Figure 14: An illustrative example of CapRL applied to natural image understanding.
25

---

Published as a conference paper at ICLR 2026
Prompt for LLM to answer questions based on Caption
will given image caption describing the visual content.
Your task is the multiple-choice question **strictly based the caption**, if the may obvious prior knowledge question wording.
Ignore any external knowledge. Do not make assumptions beyond what the caption explicitly implicitly states.
Example 1:
Cuption: <Caption Start> A woman in red coat is walking black dog across snowy park. <Caption End-
Question: What hut color is the dog?
A) Brown
B) White
-C) Black
(D) Gray
EB) Can not unswer based on the caption
The answer is
Example
Caption <Caption Start> child waving British flag during parade. <Caption End- Question: What color flag?
-A) Red
Blue
Red, white, and blue
() White
-E) Can not answer based on the caption
The unswer is E.
Now, answer the question based on the following caption:
Cuption: <Cuption Sturt> O <Caption End- Question: Q
Figure 15: Prompt for LLM to answer questions based on Caption.
Prompt for Unified Reward Model a Judge
You are presented with an image along with its generated text caption, Your task is to comparatively evaluate the caption quality based on the image.
Provide overall assessment for the caption (ruted from to S):
Overull Score: How well the caption comprehensively covers the information in the image, provides detailed descriptions of each object, and maintains accuracy in the conveyed information.
Output your evaluation using format below:
Overall Score (1-5):
Your provided follows:
Text Caption: [3]
Figure 16: Prompt for Unified Reward Model Judge.
26

---

Published as a conference paper at ICLR 2026
Prompt for Qwen2.5-VL-3B as a Judge
You presented with image along with its generated text caption. Your task is comparatively evaluate the caption quality based the image.
Provide overall assessment for the caption (rated from
Overall Score: How well the caption comprchensively covers the information in the image, provides detailed descriptions of each object, and maintains accuracy in the conveyed information.
Output overall score for this caption as single number.
Your task is provided follows:
Text Caption: [(]]
Your ovaluation:
Figure 17: Prompt for Qwen2.5-VL-3B as a Judge.
Prompt Qwen2.5-VL-72B to generate QA
Your task is generate five multiple-choice questions and their ubout the object based the provided image.
The questions should be challenge and focus the image content, you answer should strictly follow the following format:
#в*# 1. **hhich method uchieves the highest accuracy (Acc) on the FF++ (HQ) dutaset?**
A) Method "a"
H) Method "b"
C) Method "c"
-D) Ours
Answerr:** D)) Ours
#### **What the primary color kayak in the image?**
-A) Blue
-C) Black
White
** *Answer:** ** B) Red
You should strietly follow the above format and should not generate irrelevant sentences. All the quesiton should be unswered based on the image.
Figure 18: Prompt Qwen2.5-VI--72B generate QA
27

---

Published as a conference paper at ICLR 2026
Prompt for GPT5 Evaluation
You are a spocialist multimodal evaluation. Your responsibility to judge the quality of a candidate image text description strictly based on visible content provided image. assessments must rely solely on what visually verifiahle- external knowledge, assumptions, contextual guessing is permitted.
You will the description
**oour analyticul dimensions**, euch *+I-5 scale**, where higher indicate stronger performance. Every score must be supported with clear and concise reasoning.
## Evaluation Dimensions
#НН #*1, Detail Coverage**
Assesses whether the description captures the key visual elements present image.
Check whether major objects, attributes, sputial relations, and contextual cues are appropriately observed represented.
##* +*2, Content Accuracy**
Evaluates the factual correctness of the description with respect the image. Verify that all mentioned entities, colors, quantities, positions, and visual facts align with what actually visible. without introducing contradictions.
### **3, Expression Fluency**
Mcasures the clarity, coherence, and naturalncss of the language. Determine whether the description is well-structured, grammatically sound, and easy to read.
##W Succinctness Degree**
Judges whether the description conveys necessary information in concise efficient manner.
whether the text avoids unnecessary verbosity while preserving completeness and informatieeness.
#! * Scoring Scale (1-5)**
### ++1 - Critical flaws
Major inconsistencies, severe inaccuracies, missing essential content, or highly speculative interpretation.
#вय #*2 Significant d
Multiple noticeable issues; partial aligmment with the image but clearly below acceptable quality.
##R **3
Adoquate**
basic expectations; generally correct
room refinement or more precise articulation.
нu2 +*4 S
Mostly accurate, well-structured, and coherent; only minor issaes slight omissions.
##* #5 Excellent**
Comprehensive, precise, fluent, fully aligned with the visual evidence, and free of unjustified inference.
##
O*tupttt Format (ISON)**
The output format must strictly follow form below. not output anything else.
"detail coverage":
"content accuracy":
"expression fluency":
"succinctness degree":
Figure 19: Prompt GPT5 to evaluate caption
28


Reviews:
[{"review_id": "LEra2whbub", "reviewer": "Reviewer_ML2d", "summary": "This paper proposes the CapRL reframeing the captioning task as “utility for QA.” A caption is good if a vision-free LLM can answer the questions about the image using only that caption. The captioner is trained with RL using this verifiable reward (QA accuracy) in a decoupled two-stage pipeline. They start from a small LVLM (Qwen2.5-VL-3B), optimize it with their RL scheme to get CapRL-3B, and then mass-caption images to create a 5M-caption dataset used in standard three-stage LVLM pretraining.", "questions": "See above", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- Using QA accuracy as a verifiable signal rather than judge models or reference captions is well-motivated and an interesting way to improve caption quality.\n- The proposed approach shows a 3B CapRL-trained captioner matching a **72B** VL model in the decoupled QA-from-caption setting.\n- The paper provides useful analyses on (i) training-dataset size, (ii) the number of GRPO generations per step, and (iii) the number of QA pairs per image.", "weaknesses": "- Could the method incentivize VQA-specific cues rather than balanced, detailed captions? Please report caption quality directly (standard automatic metrics and a small human study) to show quantitative gains.\n- For a fair comparison, evaluate methods on the exact same image set, varying only the training approach. One approach can be evaluating a GRPO-trained VQA model that generates description → reasoning → answer on your QA dataset as a direct comparison.\n- The paper measures caption utility only via MCQs. Does the approach transfer to open-ended QA (and related tasks like grounding or summarization)?\n- It would be beneficial to include the computation required to generate the QA data, produce captions at scale, and run QA (GPU type/count, GPU-hours, throughput, wall-clock time).\n- I wonder if there is any potential bias introduced by using the 72B model to generate the QA dataset used for training.\n- Answer-model sensitivity: How much do results change when the answer LLM is swapped (different sizes/models)?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "i5JmsrP9NM", "reviewer": "Reviewer_MPnM", "summary": "The paper proposes CapRL, a reinforcement-learning framework for vision-language models that tackles the problem of reward hacking in open-ended image captioning. Instead of letting a reward model or llm judege directly score captions (which the authors show quickly collapses to trivial, short outputs), CapRL splits evaluation into two stages: (1) use a strong VL model to generate multiple vision-grounded MCQs about an image, and (2) judge a caption by asking a (cheaper) LLM to answer those MCQs using only the caption, averaging over shuffled options to reduce bias. This verifiable reward is then used to RL-tune Qwen2.5-VL variants, and the authors report that the resulting models improve across a wide suite of benchmarks.", "questions": "See weaknesses above.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "* The idea of evaluating the generated caption, not in terms of the caption itself, but in terms of how well it provides information for downstream QA seems very interesting and novel. \n\n* More QA data leading to better performance shows the scalability of the method.", "weaknesses": "[minor] In Figure 1 of introduction, the authors use the term 'Unified Reward Model as a Judge', but its not exactly clear what this is. Is it an open-source reward model trained to evaluate captionings?\n\n* In Tables 1 and 2 the authors report results on 12 LVLM benchmarks (InfoVQA, DocVQA, ChartQA, RealWorldQA, MathVista, SEED-Bench 2 Plus, MME, RW MMB, MMStar, MMVet, AI2D, GQA). Could the authors clarify what metric is being reported for each dataset (e.g., accuracy / exact match / task-specific score)? Since the main task in the paper is captioning, it would help to state explicitly how these VQA-style evaluations are computed.\n\n* When the authors mention Qwen2.5-3B + Qwen2.5-ViT, does this mean Qwen2.5-VL-3B? If not, what is the difference?\n\n* In the proposed 2-stage reward, the caption is judged by whether an LLM can answer MCQs based solely on that caption. Could you clarify how well this proxy aligns with human-oriented caption quality (fluency, succinctness, informativeness)? In other words, is there a risk that the model is being optimized for captions that are good for downstream QA rather than captions that are good for people, and if so, can the authors provide human evals or qualitative examples to show the proxy is realistic?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bRfucAKQTk", "reviewer": "Reviewer_UVsw", "summary": "This paper introduces CapRL, a framework for synthesizing richer captions to enhance VLM training. Their contributions are two-fold. 1. They incorporate the RLVR paradigm to train CapRL-3B, where the reward is defined by the average score of an LLM answering questions based on the generated captions. 2. They utilize the RL-trained model to produce an additional 3M image-caption pairs, which are combined with ShareGPT4V-1M and DenseFusion-1M to form the CapRL-5M dataset. Experiments show that pretraining on this curated dataset yields consistent performance gains over existing pretraining datasets under the same model architecture.", "questions": "Please see the above weakness section.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "1. The idea of combining RLVR + data synthesis is novel and effective, as demonstrated in the results of Table 1.\n\n2. As mentioned in appendix E, they employ a data filtering pipeline to alleviate the data-leakage issue in their pretraining corpus.\n\n3. The proposed synthesis pipeline is scalable with the model scale being only 3B.", "weaknesses": "1. While the paper claims that CapRL reduces hallucination, no quantitative evidence is provided for either CapRL-3B or models pretrained on CapRL-1M/5M. The authors are encouraged to report hallucination metrics such as POPE [1] or CHAIR [2] to support this claim.\n\n2. The experiments in Table 3 do not appear to provide a fully fair comparison. The evaluation relies on text-only captions to generate responses, whereas only CapRL has been RL-tuned specifically to produce detailed captions, giving it an inherent advantage. In real-world applications, models are typically used in an end-to-end manner rather than in such a disjoint caption–response setup. Therefore, weaker performance under this prism evaluation does not necessarily indicate poorer downstream capability. I understand the authors’ intention to demonstrate that higher-quality captions can improve downstream QA performance, but a more direct and convincing evaluation would be to apply CapRL as a post-training stage to a VLM and then compare it with other models on these benchmarks under standard end-to-end settings.\n\n3. In the QA curation process, the authors first use QwenVL-72B to generate QA pairs and then QwenVL-3B to verify that the answers are visually grounded and image-dependent. However, this filtering step may remove examples where the smaller model disagrees with the larger one, potentially biasing the dataset toward easier instances (i.e., those correctly answered by the 3B model) and excluding more challenging or informative pairs that only the 72B model answers correctly.\n\n4. It would be helpful to include captioning benchmarks and quantitative metrics to assess the improvements of CapRL-3B over QwenVL-3B, providing a clearer measure of caption quality.\n\n5. (Minor) The title and abstract may be somewhat misleading, as the paper does not present quantitative analyses of caption quality; instead, the evaluation primarily focuses on downstream task performance when using the constructed datasets for pretraining.\n\n[1] Evaluating Object Hallucination in Large Vision-Language Models, EMNLP 2023.\n\n[2] Object Hallucination in Image Captioning, ACL 2018.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2oqF7PkB01", "reviewer": "Reviewer_hugc", "summary": "This paper proposes CapRL (Captioning Reinforcement Learning), a new framework applying Reinforcement Learning with Verifiable Rewards (RLVR) to the open-ended image captioning task. CapRL defines caption quality in terms of utility: a caption is good if it enables a non-visual LLM to correctly answer multiple-choice VQA questions about the original image. The method uses an LVLM to generate a caption, and a vision-free LLM to answer questions based solely on that caption, with the resulting accuracy serving as a verifiable binary reward. Then, Group Relative Policy Optimization (GRPO) is used to train the LVLM with the reward. To support question creation, the authors develop a specific QA curation pipeline to ensure the quality of questions and answers. The authors introduce a large-scale CapRL-5M dataset generated using CapRL-3B, and demonstrate significant improvements across 12 multimodal benchmarks (InfoVQA, ChartQA, SEED, MMStar, etc.), showing performance comparable to much larger models (e.g., Qwen2.5-VL-72B).", "questions": "How does CapRL perform when the reward LLM is replaced with models of varying strengths (weaker, stronger models)?\n\nWhat is the actual computational cost of each stage in the CapRL pipeline? \n\nHow sensitive is CapRL to the accuracy of QA filtering? What happens if weakly grounded questions are included?\n\nHow does CapRL handle creative or abstract images that do not have factual QAs (e.g., artwork, memes)?\n\nDo the authors open CapRL-5M to the community?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "The core idea of redefining caption quality through its verifiable downstream utility (i.e., enabling correct VQA responses) is interesting to overcome the long-standing reward-design challenge in open-ended language generation.\n\nThe paper is well organized and readable. The methodology is clearly structured, with detailed explanations.\n\nThe dataset construction (CapRL-5M) is beneficial to the community if the author makes it open-source.", "weaknesses": "The two-stage pipeline (multiple QA samples per image, multiple-choice shuffling, etc.) introduces heavy computational overhead.\n\nThe reward signal depends entirely on a single non-visual LLM’s comprehension and question-answering ability. If that LLM is weak, biased, or miscalibrated, CapRL may learn suboptimal behaviors.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)