Dual-Stage Value-Guided Inference with Margin-Based Reward Adjustment for Fast and Faithful VLM Captioning
Authors: Ankan Deria
Adinath Madhavrao Dukre
Feilong Tang
Sara Atito
Sudipta Roy
Muhammad Awais
Muhammad Haris Khan
Imran Razzak
Venue: NeurIPS 2025 poster ()
Keywords: Inference time search
Value-Guided Decoding
Margin-Based Reward
Hallucination Mitigation
Self-Training
Primary area: deep_learning
TL;DR: ViMaR is a two-stage, value-guided inference framework that uses margin-based rewards to produce faster, more accurate, and less hallucinatory captions, enabling scalable and self-improving vision–language models.

Abstract: Despite significant advances in inference-time search for vision–language models (VLMs), existing approaches remain both computationally expensive and prone to unpenalized, low-confidence generations which often lead to persistent hallucinations. We introduce \textbf{Value-guided Inference with Margin-based Reward (ViMaR)}, a two-stage inference framework that improves both efficiency and output fidelity by combining a temporal-difference value model with a margin-aware reward adjustment. In the first stage, we perform a single pass to identify the highest-value caption among diverse candidates. In the second stage, we selectively refine only those segments that were overlooked or exhibit weak visual grounding, thereby eliminating frequently rewarded evaluations. A calibrated margin-based penalty discourages low-confidence continuations while preserving descriptive richness. Extensive experiments across multiple VLM architectures demonstrate that ViMaR generates captions that are significantly more reliable, factually accurate, detailed, and explanatory, while achieving over 4$\times$ speedup compared to existing value-guided methods. Specifically, we show that ViMaR trained solely on LLaVA Mistral-7B \textit{generalizes effectively to guide decoding in stronger unseen models}. To further validate this, we adapt ViMaR to steer generation in both LLaVA-OneVision-Qwen2-7B and Qwen2.5-VL-3B, leading to consistent improvements in caption quality and demonstrating robust cross-model guidance. This cross-model generalization highlights ViMaR's flexibility and modularity, positioning it as a scalable and transferable inference-time decoding strategy. Furthermore, when ViMaR-generated captions are used for self-training, the underlying models achieve substantial gains across a broad suite of visual comprehension benchmarks, underscoring the potential of fast, accurate, and self-improving VLM pipelines.

Code: https://github.com/ankan8145/ViMaR

Dual-Stage Value-Guided Inference with Margin-Based Reward Adjustment for Fast and Faithful VLM Captioning
Ankan Deria', Adinath Madhavrao Dukre', Feilong Tang', Sara Atito?, Sudipta Roy Muhammad Awais", Muhammad Haris Khan', Imran Razzak
Mohamed bin Zayed University of AI, Abu Dhabi, UAE
University of Surrey, UK 3 Jio Institute, India
UNSW, Australia
Abstract
Despite significant advances inference-time search for vision-language models (VLMs), existing approaches remain both computationally expensive and prone to unpenalized, low-confidence generations which often lead to persistent hallu- cinations, We introduce Value-guided Inference with Margin- based Reward (ViMaR) a two-stage inference framework that improves both efficiency and output fidelity by combining temporal-de differeeceee value model with margin- aware reward adjustment. the first stage, we perform a single pass to identify the highest-value caption among diverse candidates. In the second stage, we selectively refine only those segments that were overlooked or exhibit weak visual grounding, thereby eliminating frequently rewarded evaluations. A calibrated margin-based penalty discourages low-confidence continuations while preserving descriptive rich- ness. Extensive experiments across multiple VLM architectures demonstrate that ViMaR generates captions that are significantly more reliable, factually accurate, detailed, and explanatory, while achieving over 4x speedup compared existing value-guided methods. Specifically, we show that ViMaR trained solely on LLaVA Mistral-7B generalizes effectively guide decoding in stronger unseen models. To further validate this, we adapt ViMaR to steer generation in both LLaVA- One DneViiion-Qeen---i and (0wen2.5-VL-311 leading to consistent improvements in caption quality and demonstrating robust cross-model guidance. This cross- model generalization highlights ViMaR's flexibility and modularity, positioning it as scalable and transferable inference-tine decoding strategy. Furthermore, when ViMaR- -eeneraeee captions are used for self-training, the underlying models achieve substantial gains across broad suite of visual comprehension benchmarks, underscoring the potential of fast, accurate, and self-improving VLM pipelines.
1 Introduction
Vision-language models (VLMs) 6 17 23
have revolutionized our ability to produce fluent,
richly detailed image descriptions. However, they still contend with two intertwined challenges: generating precise, fine-grained captions and combating "hallucinations" [21 6 47 36 46 23 37], which often arise from unpenalized, low-confidence outputs that misrepresent the scene. Simply scaling up training data can ameliorate these issues, but it incurs prohibitive annotation and API costs,
Codee http:///ichuu..cccm/aaaaaaa/////
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

making it fundamentally unscalable. Meanwhile, standard decoding strategies such as best-of-N or greedy sampling either underutilize the model's representational power or demand exhaustive. compute-intensive re-scoring of every candidate at each generation step R3
32 50 have demonstrated that inference- time search, using a pretrained process-reward model to iteratively refine candidate outputs can substantially elevate response quality and even generate synthetic data for further model training [8 [54]. Extending this paradigm to VLMs, however, introduces unique challenges: unlike text-only tasks, VLMs require a reward signal that captures both visual grounding and linguistic coherence across multiple sentences. To address this, Wang er al. [46] proposed the Vision Value Model (VisVM), which employs temporal cempora--differncce
Recent advances in large language models (LLMs) 28
difference learning [35] over CLIP similarity scores estimate the long-term quality of each candidate sentence. Though, VisVM markedly reduced hallucination and enriches visual details, however scoring every candidate at each step incurs considerable overhead.
To overcome aforementioned challenges, we present a two-stage search pipeline that preserves or improves caption fidelity while reducing end-to-end inference time by over 4x. In the first stage, a single 'nbest-of" pass selects the highest-value caption (coarse caption) under our trained policy, In the second stage, we propose perform targeted refinement (fine caption): additional search is restricted solely to those segments where salient image regions were likely overlooked, thereby eliminating the need to re-score the full candidate set on every generation step. At the core of our method lies new margin-basee reward adjustment for training: whenever a candidate's 's CLIP similarity falls below calibrated threshold, we impose a penalty proportional to the gap. This mechanism seamlessly integrates into the existing temporal difference framework, sharpening the model's preference for factually grounded, detail-rich phrases. In results, ViMaR demonstrates strong cross-model generalization: despite being based on LLaVA Mistral-7B, ViMaR effectively Vision-
guides decoding on other stronger models such as LLaVA the generalizability and scalability of our framework, making it suitable as plug-and-play decoding strategy across diverse VLM architectures. Furthermore, by leveraging ground-truth captions in our dataset, we ensure that truly accurate descriptions receive the highest reward signals. We validate our approach in two comprchensive studies. First, in COCO-based descriptive captioning. our optimized value-guided search produces captions that markedly more detailed and substantially less prone to hallucination than those from VisVM search, greedy decoding, best-of-N or standard CLIP-PRM search. In blind evaluations conducted using both GPT-4o and human judges, our captions are preferred in 49.3% and 64% of pairwise comparisons, respectively, against outputs generated by the VisVM (see Figure [].. These performance gains achieved at a much lower cost, as our inference pipeline runs considerably faster than the state-of-the-art Vis Second, we leverage these high-quality captions to self-train the base LLaVA-NexI-7B model. Fine-tuning on our optimized-gennratee data yields consistent improvements across eight diverse multimodal benchmarks, achieving an average performance uplift of 15. 5.877. Together, these results highlight the potential of a fast, accurate, and computationally efficient VLM inference paradigm that supports celf-improvement.
B. This highlights
2 Related Work
Early joint vision-ter text models combined convolutional or transformer- based image encoders with sequence decoders to tackle tasks such as object tagging, image captioning, and visual question answering [5 18 [1). More recent approaches fuse large pretrained language backbones with powerful visual representations (e.g., CLIP) to enable instruction following, in-
Visgon-Language Modeling:
context multimodal reasoning, and zero-shot generalization [ [ [43. Despite these capabilities, VLMs remain prone to hallucination, producing confidently stated but incorrect content [3] 14 30).
Hallucination Mitigation: Hallucinations in languaele models are typically addressed in the following ways. Enhance the quality of supervised fine-tuning (SFT) datasets through human annotation, synthetic caption rewrites, or contrastive filtering to provide more accurate grounding [12 or apply corrective methods during post-trainin post-training, such as fine-tuning with adversarial negatives, consistency checks, or calibrate self-rewarding to detect and suppress spurious phrases [2] In contrast, our approach leaves model weights and training data unchanged, instead devising a two-stage inference-time search to actively reward well-grounded descriptions while penalizing low-confidence outputs that are prone to hallucination.
9, 44, 45],
34 55
26

---

Descriptive captioning extends single-sentence models to produce multi-sentence paragraphs that comprehensively describe both global scene context and fine- regional details [ 5 Early work on paragraph captioning demonstrated that standard sequence models often generate repetitive text with limited diversity, motivating approaches that explicitly promote novel content and discourage redundancy [16] 27]. Reinforcement learning methods such as
Descriptive Paragraph Captioning:
Self-Critical Sequence Training have been adapted to optimize non non-differentiable paragraph-Ievel metrics, improving coherence but still requiring heavy sampling during inference [29]. Partially non-autoregressive architectures further reduce latency by updating only segments of the caption in
parallel, yet they can struggle to maintain sequential consistency across sentences [3]. To mitigate low-quality outputs, DeepSeek VL2 employs a lightweight quality control pipeline powered by DeepSeek Chat to quickly score and filter captions based writing quality alone, effectively pruning imprecise or bland descriptions before post-training 48 20 More recent strategies incorporate human-style feedback, for example, caption reformulations, to refine output at inference time, achieving gains in factuality and readability without additional supervision [4]. Nevertheless, these methods typically rerank or regenerate full paragraphs at each step, incurring substantial compute. In contrast, our approach targets the inference- search itself: By unifying a two-stage best-of pass with targeted segment refinement and a margin- baaedd penalty for low-confidence phrases, we obtain richly detailed paragraphs with minimal extra computation.
Inference- Time Search: petreence-imee search has emerged powerful mechanism for enhancing model outputs in domains ranging from code generation and mathematical reasoning to multimodal planning and robotics 31
g In the text-only setting, techniques such as controlled decod- consistently enhance performance by using learned process value model to rerank multiple candidate outputs. The effectiveness of inference-time search depends primarily on the process reward model (PRM), since the accuracy of its reward signals directly affects both the quality of the generated responses and the computational cost required to obtain them.
40
54
ing 88, Best-of-N sampling 7 19, and Monte Carlo Tree Search 38 42
Translating these ideas to vision-language models (VLMs) introduces unique challenges: the reward signal must capture both risual-text alignment and sequential coherence across sentences. Zhou er al. (55) first explored ILIP-based scoring as proxy reward, using positive and negative sample mining to refine the model post hoc. Xiong et al. [49] proposed LLAVA- -Critic, which evaluates entire paragraph-level captions filter out poor outputs, but lacks stepwise granularity. Most recently, Zhang et al. [54] introduced Rest-MCTS* employing process-reward-e -eewdddgguided tree search to iteratively refine multimodal responses, yet still incurs quadratic inference cost the tree grows. Building on lemporal-difference -ifffrrene value learning in VLMs, Wang et al. [46) presented the Vision Value Model (VisVM), which estimates both immediate and future sentence value via CLIP similarity and steers search toward low-hallucination. high-detail trajectories. Although VisVM substantially elevates descriptive quality and reduces hallucinations, its naive implementation must re-score all N candidates at each generation step, resulting in an O(N x S) inference overhead (where S is the number of sentences in each step). To mitigate these limitations, we developed ViMaR, a two-stage inference framework that preserves long-term value signals while improving efficiency. ViMaR conducts a best-of pass followed by targeted refinement, applies margin- based penalty to reduce redundant scoring of every candidates in training, and uses beam search for stable, diverse This achieves over 4x faster inference while maintaining or improving caption quality.
decoding 77.
3 Value Guided Inference Framework-ViMar
We formulate the VLM captioning process as sequential generation task modeled by policy m over probability distribution pa. Given an input pair consisting of textual prompt and an image I, the model produces a multi-sentence caption [Mo: V/, 1/2,... ..m,, where Yo is the first step caption and each Y denotes a sentence-level output. At the first step, the model produces yo by sampling from yo- ~ po # x I). while each subsequent sentence yıoo is drawn conditionally from Wioo po x, I, L,yc)) followed by evaluation and potential selection at each step. We cast this caption generation process as a Markov Decision Process (MDP) defined by the tuple (S, R, 7), where each state 8; E S consists of the prompt-image pair (ar, 7) and the sequence of previously generated sentences yeci, and the action y/ E A transitions the model to the next state 8i+1- The reward function R(s;, y/., parameterized by a value model Vp. scores the quality of the generated output at each step, while the discount factor y E [0, 1] governs the trade-off between immediate
3

---

and future rewards. This MDP formulation enables inference setrence-timeeeeee search to explore alternative trajectories and prioritize high-quality, visually grounded captions through value-guided decoding.
3.1 ViMaR Training
Our proposed model, ViMaR, is designed to estimate the long-term utility of image-conditioned sentence candidates, accounting for their potential to influence subsequent generation steps. We adopt lemporal-difference (TD) learning strategy [35], which enables ViMaR to recursively refine its predictions of the cumulative reward from any given state s; (ys, I), where y: is the current sentence and I is the input image.
Training Method:
Given a training triplet (yt, Vo+1. I) (the current and next sentence in a paragraph, together with the associated image), we first compute the similarity score á between y/ and the image using a pretrained process reward model (PRM). To discourage low-confidence or potentially hallucinatory outputs, we introduce a margin- reward adjustment. The reward rss, at each state is computed as:
88 if 2 T Tsc 1 - otherwise
(1)
Here, T denotes a calibrated threshold that serves as a margin for penalizing uncertain or weakly grounded predictions. When the PRM score falls below this threshold, a negative penalty proportional to the margin gap is applied, encouraging the model to avoid such candidates during search.
The model is trained to minimize the discrepancy between the predicted value of the current state and the target value, which is defined as the sum of the immediate reward and the discounted value of the next state. Formally, the training objective is:
C(p) = B(u. [tr.. + +Voovi4,, - Volu,,
Here, Vp is the value predicted by ViMaR, is the discount factor, and denotes the learnable model parameters. The training set comprises image-caption pairs segmented into sentence-level transitions to capture both local grounding and long-term contextual dependencies.
To train ViMaR, we construct training triplets of the form (yi, (Wt, U1+ T), where yi is a sentence from a paragraph-level caption, Vi+1 is its immediate successor, and is the corresponding image. These triplets are derived from multi-sentence image descriptions y = [y1+ 12.-.... . Vn paired with their respective images. Modeling the long-term value of a sentence requires capturing not only its direct alignment with the image but also downstream influence on the continuation of the caption. To ensure a diverse set of generation patterns, we begin with 23K images from the COCO 2017 training split and pair them with detailed prompts from the LLaVA-150K dataset. For each image prompt pair, we include both the ground-truti caption and five additional captions generated by a VLM using a mix of greedy decoding and remperature-contrre controlled sampling to promote diversity. Each paragraph is then segmented into ordered sentence pairs, yielding a total of 792K triplets. We used 732K examples for training and 60K for validation.
Training Data:
Implementation Details:
We build ViMaR on top of the -ea.--eext-Mistral-- 7B architecture. Concretely, we attach linear value head to the penultimate transformer layer; this head outputs scalar estimate of the cumulative, long-term reward for each image-sentence in ViMaR are initialized from the pretrained LLaaA--ext-Mistral--7 checkpoint and remain trainable alongside the new value head. For the process-reward model (PRM), we choose the CLIP-ViT. This
sentence state. All other weights
choice offers two advantages: (1) CLIP's image-text embedding similarity provides a proven metric for visual grounding, yielding reliable reward signals for descriptive captioning; and (2) leveraging the native CLIP-VIT avoids external dependencies or costly human annotations, creating fully self-contained training pipeline and easily customized with our margin-based reward adjustment. To support penalization and reduce hallucination, we modify the Case-based PRM by introducing margin- based reward adjustment (as described in Section 3.1 thereby downweighting low-confidence alignments during reward computation.
3.2 Inference-Time Search with Two-Stage Refinement
Once trained, ViMaR serves as a value model Vp to guide inference-time search, enabling the VLM to produce more accurate and visually grounded descriptions. In the first stage, we perform

---

Algorithm Two-Stage Inference-Time Search with ViMaR
Require: Test sample {z, 1), VLM policy po. value model Vp temperature list {Tx 1N candidate count K
Output: Final response y
1: # Stage 1: Generate diverse base captions
2: Initialize candidate set
3: for Tn T do
4: Generate K paragraph-Ieve responses {y l 2,
5: Add all [y(m to C
6: Select base caption y" = arg yax VV !)
7: Stage Add supplementary segments to improve grounding
8: Identify under-grounded or missing visual regions in y²
9: for while Generation is not Done do
10: Initialize candidate set S;
11: for Tn E T do
12:
Generate candidate sentences {s) (8| sk=1 ~ pol. x,, M,ee Tr)
(n,A)
Add all (s/mxk) S;
14: Select hest sentence 8 arg max I)
Append st to y" at the appropriate position return final refined response y
16:
pall-paragraph generation using beam search over the entire prompt-image pair (z, 1), applying temperature sampling with N distinct decoding temperatures (Tr) For each temperature T,, the model samples K complete paragraph-level candidates from the policy: y po( # xx, I,T), where # denotes the end-of-caption token. This results in a total of N K candidate captions. Each is scored holistically by the value model (y Z). and the caption with the highest predicted value is selected as the base output. In the second stage, we perform targeted refinement on the selected base caption. For each segment Y/ with low visual grounding missing objects, resample alternatives from the conditional distribution yi ~ po x. I, yc, T,), drawing K candidates per temperature over N temperatures for total of N x K alternatives. Each candidate is scored by the value model (B). I), and the highest-value sentence is incorporated into the caption. This refinement loop repeats until all salient content is addressed and an end-of-sequence (EOS) token is generated. This two-stage search preserves long-range reasoning while selectively improving weaker segments, reducing inference cost without losing detail or accuracy. A complete overview is provided in Algorithm[]
Experiments
In this section, we empirically evaluate riMaR-guided inference-time search framework. Our investigation is centered around the following key questions: (1) Does the proposed two-stage decoding strategy, guided by the learned value model, generate more accurate and visually grounded outputs compared to existing inference-time decoding methods? (Section 4.1) (2) Can ViMaR's refined outputs serve as high-quality supervision signals to enhance the visual comprehension
capabilities of VLMs through self-training? (Section 4.2) (3) How efficient is our method in terms of
inference speed compared to baseline search strategies? (Section
4.1 Evaluating the Effeetiveness of ViMaR-Guided Two-Stage Search
Baselines and Implementation Details: We compare our two-stage ViMaR-guided search against four established inference-time decoding strategies, built on Decoding: Stepwise selection of the highest-probablns probability token. Best-of-N (BoN): Generate 30 full captions using five temperatures {0.1, 0.3, 0.5, 0.7, 0.9) (six per temperature) and choose the best via GPT-4o. CLIP-PRM Guided Search: Stepwise search using CLIP-VIT similarity as the 6 samples per temperature. VisVM-Guided Search: Single-stage inference-time search guided by the Vision Value Model, evaluating all N K candidates at each sentence step. ViMaR Two-Stage Search (Ours): Stage generates paragraph candidates with (N 5, K 6) and selects the best by Vp: Stage 2 refine and add additional
Greedy
reward, with temperature decoding (N = 5) and
5

---

segments to improve the caption details. All methods employ Lit-A---xxt-MMstralll as the base VLM and initialize ViMaR's value head from its penultimate layer. We fix the temperature set to {0.1, 0.3, 0.5, 0.7, 0.9} and sample K = 6 candidates per temperature in both stages. CLIP-VIT (the native VL encoder) serves as the PRM for consistency and cost-efficiency. In all experiments, we kept the total decode calls identical to ensure a fair comparison of quality versus compute.
Two-Stage Value-Guided Search Enhances Caption Quality
To evaluate the effectiveness of proposed search strategy, we sample 1,000 images from the COCO Train2017 dataset and pair each image with the prompts from the LLaVA- detailed description dataset, resulting in 1,000 maaaee-rrmmp pairs for evaluation. We generate one descriptive caption per pair using our two-stage value-guided decoding strategy and four alternative decoding inclodi--iclluding greedy decoding, BoN search, CLP-PRM sampling, and the original V- guided search. The quality of the generated captions is assessed through both human preference studies and automated metrics. For human evaluation, pairs and ask annotators to compare outputs from our method against each baseline, identifying the preferred response in each case. As shown in Figure Tal our two-stage strategy consistently outperforms all baselines, achieving win rates of 64.0%, 63.3 %, 66.0% and 69.7% over V-- guided search, CLIP-PRM, BoN and greedy search, respectively. Notably, greedy decoding performs the worst, while Vis -iggiddd search offers meaningful improvements- still lags behind our approach, highlighting the benefits of long-horizoo and localized refinement. As illustrated in Figure 2. our method generates descriptions that are both richer in detail and better aligned with visual content. For instance, descriptions include nuanced elements such as "clearly raining in the image", which are often omitted by competing methods. In addition, we evaluate model outputs using GPT- 4o-based pairwise comparisons. Figure [ indiiateeee that captions generated with our two-stage method are preferred in 49.3%, 68.4%, 65.4%, and 73.8% of the cases over the same four baselines. These findings demonstrate that our search strategy improves both the fidelity and richness of generated descriptions, pushing the boundaries of VLM visual comprehension.
randomly select 300 paaagg-prompp
OUP Guilet
1377
22:2%
35%
11.44
Wint
(a) Human evaluation of win rates.
(b) GPT-4o evaluation of win rates.
Figure 1: Comparison of -eseedddseiiition quality across search strategies for LLaVA-Next-7BB (a) Independent human raters corroborate these findings, selecting ViMaR-guided outputs at significantly higher rates than all other methods. (b) Win-rate judgments by GPT-40 show that ViMaR-guider search consistently outperforms all other search methods.
2 Two-Stage Value-Guided Search Mitigates Visual Hallucinations
To assess the impact of our two-stage value-guided search on visual hallucination. we conduct quantitative evaluation using 500 randomly sampled images from the COCO Va12017 dataset. Each image is paired with detailed prompts sourced from the -LLVA-1 10k dataset. We employ two widely used metrics to measure hallucination: CHAIR (30 and MMHal 34. The CHAIR metric quantifies hallucination at both the object level (CHAIR)) and sentence level (CHAIRs) as follows:
CHAIR;
[hallacinated objects) ||
CHAIRs = (captions with hallacinated objects]
[[ mentioned objects]] 1
[[ll captions]) }
In addition, we use MMHal 34] a multimodal hallucination benchmark that evaluates object-level
consistency using a fine-grained image-text alignment model. Table[rrepott the hallucination results across different inference-time decoding strategies. Our two-stage value-guided search achieves significant reductions both CHAIR and MMHal hallucination rates, outperforming all baselines and Vis -Mguuded search.

---

Table 1: Comparison of visual hallucination and inference efficiency across decoding methods on the COCO Val2017 dataset. CHAIR and MMHal assess ballucination quality (t / indicate better performance), and Avg. Approx Time reports the average inference time per sample seconds. Top-performing results are highlighted in bold.
Method
CHAIR;
CHAIR; J MMHal
MMHal Rate
Avg. Approx Time
Greedy Decoding
32.4
5.9
2.94
0.52
62 s
BoN
3.06
01.45
CLIP-Guided
28.4
5.5
2.96
0.49
286
VisVM-Guided
26.2
4.6
3.30
0.39
462
VIMaR (Our)
23.1
4.1
3.75
0.35
108
These results demonstrate that our method effectively reduces visual hallucinations during caption generation. Notably, even though method operates under a smaller decoding budget compared to methods like Visvm, it still yields superior performance. This highlights the efficacy of our localized refinement strategy, which selectively targets visually ambiguous segments for re-generation. Our improvements align with the design of the underlying value model, which is trained to predict long- term rewards using TD learning. By scoring candidate continuations based their expected future quality, including grounding fidelity, our model encourages selections that reduce hallucinations throughout the entire sequence.
Table 2: Evaluation of ViMaR ViMaR-guided decoding on visual comprehension benchmarks for both LLaVA-Mistral and LLaVA- Vision-Qwee models. Our two-stage inference framework consis- tently improves performance across all evaluated tasks, highlighting its effectiveness in enhancing output fidelity and visual grounding. Compared to the base models, ViMaR yields consistent gains, with an average improvement of 15.87% computed across all evaluation benchmarks, including normalized variants of CHAIRs, CHAIRi, MMHal (normalized as 100-CHAIR, 10 (O-CAARi,, and
I I-MMHal), and others.
Visual Comprehemsion Benchmark
Hallucination Benehmark
MM-Aer
XMBenehh
1MMMU
Math
CVBench
IJ..VA W
StM
CHAIRs
CHAIR
MMHta
Base
SFT Data Source
Avg.
Original
38.5
36.0
5.9
0.52
Greedy decoding
43.5
74.6
34.9
37.8
66.2
75.
36.7
33.2
6.3
2.97
0.54
-1.44%
CLIP-HoNN (6)
42.8
76.2
35.2
39.7
63.8
74.8
35.5
29.7
5.2
3.05
0.48
+2.45%
GPTH-BoN (30)
35.4
40.9
36.9
5.4
-4.82%
CLIP-PRI search
46.1
75.8
35.8
39.6
68.5
78.1
37.6
26.0
5.2
3.01
0.50
+5.33%
VisVM search
48.3
76.7
36.1
42.1
69.8
78.4
38.0
22.6
4.3
3.26
0.44
+1..8%%
ViMaR
49,8
37.4
+15.87%
LLaVA--Oherissoon-nee---
Original
58.8
81.7
47.3
56.1
86.9
VIMaR
60.5
84,8
49.4
56.9
80.6
88.5
62.6
15.3
3,96
0.34
Qwen2.5.VI..88
Original VIMaR
61.8
79.1
31.5
62.3
72.1
88.4
55.9
18.5
3.82
0.32
62.3
81.2
33.4
64.5
72.8
89.2
56.3
17.2
3.2
3.94
0.28
+6.7 %
4.2 Self-Training Vision-Language Model
Beyond its utility at inference time, our two-stage value-guided decoding method offers a compelling opportunity for self-training, leveraging high-quality the visual reasoning capabilities of lasion-language models (VLM). This section investigates whether the captions produced by our method can serve as effective supervision data for instruction tuning.
responses to further enhance
Training Setup: We construct our supervised fine-tuning (SFT) dataset using the same 23,240 <image, prompt> pairs used for training the value model (as detailed in Section (.I). Applying our two-stage value-guided decoding strategy, we generate descriptive caption for each pair, resulting in 23,240 <image, prompt, response> triplets for downstream training. All models are fine- tuned starting from the LLaVA--eet-Mistra-- comparison, we adopt the same dataset, evaluation metrics, and scoring setup used in the original VisVM paper. Full-parameter fine-tuning is conducted using a learning rate of le-6. We directly compare our approach against the following baselines: greedy decoding, CL.P--based Beam-of-N (BoN), CLIP-PRM guided search, and Vis M-guided search. This evaluation allows us to assess the effectiveness of our search method not only at inference time but also mechanism for
checkpoint. To ensure a rigorous and consistent
7

---

generating high-quality supervision signals that improve the base model's visual comprehension through self- self-training.
Evaluation Benchmarks: We evaluate our method across two categories: (i) Visual compre- hension, using seven established benchmarks, including Vet [52) MMBench (24], MMMU [53], Math Vista [25], CVBench [39], LLLVA-Willd [22), and MMStar [0); and (ii) Hallucination
analysis, assessed via CHAIR 30) and MMHal 34 metrics. These benchmarks collectively measure
the accuracy, reasoning, and visual grounding quality of the generated responses.
Visual Comprehension Results: Table summarizes the performance of LLaVA-Next-Ti after fine-tuning on captions generated by different inference-time search methods. With the exception of the greedy decoding baseline, which shows marginal declines in some tasks, all self-trained models exhibit improved scores on the suite of visual reasoning benchmarks. Notably, our two-stage value- guided search achieves the most pronounced improvements. with an average uplift of 15.87% relative to the base model and 4.79% over the VisVM. These gains markedly outperform those achieved by Best-of-N, CLIP-PRM, and VMs search, underscoring the exceptional effectiveness of our generated captions as high-quality supervision for advancing VLM visual comprehension.
Visual Hallucination Results: As presented in Table 2 our proposed two-stage value-guided search demonstrates substantial improvements in mitigating visual hallucinations within LLaVA- Next. When evaluated across four hallucination metrics drawn from CHAIR and MMHal benchmarks, our method achieves relative reduction of 30.87% in hallucination rate. This clearly surpasses the improvements observed with CLIP-BoN (9.46%), and VisVM search (20.91 %). These outcomes confirm the robustness of our search strategy in generating responses that are not only detailed but also grounded more accurately visual content.
GPT4o-BoN (7.82%). CLIP-PRM search
Cross-Model Generalization While ViMaR is trained solely using outputs from LLaVA Mistral- 7B, we evaluate its performance when applied to the stronger LLLVA-One Vision-Qwen2-77 and Qwen2.5-VL models. As shown in Table ViMaR search yields consistent gains across multiple benchmarks, improving the average score across all benchmarks. These results demonstrate that our value model and refinement strategy generalize effectively across architectures, highlighting ViMaR's flexibility and plug-and- and-play play applicability in high- perforeeformin VLMs.
Toward Self-Improvine Improving Vision-Language Models: The results highlight the potential of our approach as a self-training paradigm for dision-languag models. Importantly, the entire pipeline constructed without the need for external supervision or third-party models: our value model is trained using the CLIP encoder embedded in LLaVA- and initialized with its parameters. The supervised fine-tuning data are generated by leveraging our own inference-ting search strategy with L.LaVA-Nexs ensuring that all learning signals originate from the model itself. This closed- loop design sets the foundation for future extensions of self-training VLMs, enabling continual performance enhancement without additional human annotations or external models.
4.3 Inference Efficiency
We evaluate the efficiency of our two-stage value-guided search in terms of average inference time per sample and compare it with several existing decoding strategies, as summarized in Table T While achieving state-of-the-an art performance in reducing visual hallucination, ViMaR remains highly efficient, requiring only 108 seconds per sample on average. In contrast, BoN, CLIP-guided, and VisVM-guided scarches incur significantly higher inference costs (668s, 286s, and 462s, respectively) due to exhaustive scoring or step-by-step evaluation at each generation step. Although greedy decoding is the fastest (62s), it performs the worst across hallucination metrics. ViMaR achieves strong balance between quality and compute, reducing hallucinations substantially while remaining nearly 2.6x faster than CLIP search, 4.3x faster than VM, and over 6x faster than BoN, making it well-suited for practical deployment.
Observations and Limitations
To better understand the behavioral differences between our decoding strategy and VisVM, we analyze a representative case where both models are tasked with generating captions for the same
8

---

you deseribe asaln features image
Attestion May aflee seutence gencrnted
to glven frage a group of people are resigaing many day on sttewak....
2212
ViMaR Guided
T250
X
2,37
insque captares unenest in many day....
tie leart of day; these people
VisVM Guided
dloog
Response penerated by different models
ViMall Gardeed
proup
day addealk It's clieity nening
people urcherlbe and lhe grovnd looks The image 2axh of hem holding umstenlaa bucidrop, bolling black umtrells, blending with the while the
ground bersalh their dlikk wilh relleting musd colors of the shorld tarzssetves maotle another
Theee indivillale ar
the
belding bacc and whtrre chechered umbentaa with bise bacipeci,, wasch smods the thind
holding viteii urbeeill,, adding pep ootor the On ledt wexttng
acrouzcings The people desssed appeopeately the weatbrr The sidewali (beyter walking lined with aod tlads The anuder tuen briling, sabiee archstectuml exement Ovenil, image copttres typcal may oehon with people gonng sbost
the moldte
gayy jacket malks past heeps weather
for weating wabe back
shonider beg peoviding
hotigprtdd, wittout unirella,
blooi
Gusted
paity day, a group people makknng way along a pulh each amoed with unteells stied thempelve drveppoar The person unbtotlla gray pidort, pertape indeningg unpeetts adding a pop
carrying backpeck, diazaed jacket also carryng a tractpuck, sseds
camanalage
gray the panses
min-ssized summnddnngs. path they palatte day: ngght grezs the patt, wnitng nest cocapon: sees,
another panaes
contrating
sanktt ino fe disroos porsiing a seeeze backstop this eveyray scece, green besch sis quaetly bssile and life, these faree intivudaals sovigale way thnough
sbelter proective eanapy: image captune moment ermpriiy their respetive untoe.las.
run, engrossed their own world under
CLIP- PRA Guited
The imags suptaro mexuccat as mainy day. Thase are indivislass in fie fisme, anch bolding sns unsteelle to shicid derrchees from the Statting fiors fhr iefl,
Fiod penverra is bolkinng u and clackeool undeclli They alling the wall finze decseol by unduclla Nest tham, the sosuall pesses hubding gees, bolding lack untrrill..
vilrass and unitolli thay are alar walking on sidecsik, their iilckes under untrilla the thind prsan, standing
shir and htack parts, stardiing black paats, standing still riia founts paraot. standan the adowalk. bolling Nack subrrlla. they see wearing the mun beckgrourd, tharch grace bunch and troe, adding in the merall streophere of image. The nit grase and the people with thiir aniicillo arggst that bass caining recaatly.
Taey are wancing a white
Grewdy Deccoting
In the imass a gropp of people are seen wxllinng on a videorale dering miny day. The day aoov then is momass, casting a son light the some The ground bernzah thiir foet is wat, rellading the gray of the The firs gersen is talkiing rod unirrllaa which stunca out agains the mual colers the scene. The secoend ganon is la lokking: black mnstoolls, griviiing work contree to red oae The thin perar is zse huiling in unbriitt, but imsead is woring a witii misease that matchee the inrrradd siy. The siciwalk the are walking in is itaal with trus and grae, abling a toach of saloow to the ahea watling In tee backgoound. thee's a boilding with goost son", aiiling motthe "quach of color the soane The people are movin in the same derector, suggenting try might be beadag the same destratoos. Ther actons and the oryacts they carry auggast that may are prouaved for the miny wenther. Ovanil, this irage espiires a moment of everyday hie in city dunng man wather.
Figure 2: Qualitative comparison of decoding strategies. The top section shows how our value model and VIsVM select different candidate responses, along with the resulting changes in LLaVA- Next's attention maps. Notable shifts in attention distribution highlight the influence of value-guided selection. The bottom section presents full captions generated by various search methods. Correctly grounded details are marked in green, while hallucinated content is highlighted in red. Our two-stage decoding strategy produces responses that are more accurate, detailed, and visually aligned.
image and prompt. As illustrated in Figure 2 L.L generates three full-sentence candidates. Vis selects the final sentence based on local scoring, choosing the third candidate with the highest immediate score (2.285). In contrast, ViMaR evaluates each candidate in the context of the full generated caption from the first stage, considering its broader contribution to overall caption quality. As result, our model selects the first candidate, which, despite a lower local score, yields the highest predicted global value (2.342) due its better grounding and potential to lead to more coherent and accurate follow-up content. This example highlights how ViMaR's 's objective function produces more discriminative and globally aligned scores (e.g.. 2.342 vs. 1.638), in contrast to VisVM's closely clustered local scores (e.g., 2.254, 2.216, 2.285), which limit its ability to distinguish high-quality candidates and occasionally result in hallucinated or less grounded outputs.
The lower portion of Figure 2 displays the full captions produced following these selections. Our two-stage value-guided approach produces descriptions that are richer in detail and better aligned with the image. For example, it correctly preserves nuanced visual elements such as "left is holding black and white checkered unbrella with a blue backpack," " while avoiding common hallucinations, such as misattributing visual attributes to the wrong individual. In addition, we visualize the image-text cross- maps corresponding to the two selection paths. The attention map from ViMaR shows broader and more balanced coverage of the scene, reflecting the model's ability to incorporate global context and peripheral visual details. Overall, this case study highlights the core distinction between the two approaches: while VMs performs local step-by-step selection based on immediate reward estimates, our two-stage method first analyzes entire captions to select the most globally coherent candidate, followed by targeted refinement of under-grounded segments. This global-to-local strategy

---

leads to more informed decisions and ultimately more accurate, grounded, and comprehensive descriptions.
6 Conclusion
We introduced ViMaR, a two-stage value-guided inference framework that enhances both the efficiency and factual accuracy of vision-language model decoding. ViMaR integrates temporal- difference value model with a margin-based reward adjustment to selectively refine low-confidence weakly grounded segments, thereby reducing the computational cost associated with conventional search methods. The framework delivers substantial improvements in caption quality and hallu- cination mitigation. while achieving significantly faster inference than existing aalue-guided and search-based decoding strategies.
Comprehensive qualitative and quantitative evaluations demonstrate that ViMaR exhibits strong cross-model generalization. value model trained solely on LLaVA Mistral-7B effectively guides generation in the more capable LLaVA- Vision- Sisiion-wwe2--BB 7B and Qwen2.5-VL. VL-3B models, high- lighting the scalability and modularity of our inference strategy. Furthermore, when ViMaR-generall captions are used for self-training, the underlying models achieve consistent gains across diverse suite of visual understanding henchmarks. Overall, ViMaR establishes a fast, accurate, and general- izable decoding framework that advances visual language generation and lays the groundwork for scalable, self-improving vision-language models.
References
[1] Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, lain Barr, Yana Hasson, Karel Lene, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems, 35:23716 2022.
[2] Jinze Bai, Shuai Bai, Shusheng Yang. Shijie Wang. Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou, Qwen-vl: A frontier large vision-language model with versatile abilitics. arXiv preprint arXiv:2308. 12966. 1(2)3, 2023.
[3] Zechen Bai, Pichao Wang, Tianjum Xiao, Tong He, Zongbo Han, Zheng Zhang, and Mike Zheng Shou. Hallucination multimodal large language models: survey. arXiv preprint arXiv:2404. 18930, 2024.
[4] Uri Berger, Omri Abend, Lea Frermann, and Gabriel Stanovsky. Improving image captioning by mimicking human reformulation feedback inference-time. arXiv preprint arXiv:2501.044 00.5511,,13,,, 2025.
[5] James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li. Long Ouyang, Juntang Zhuang, Joyce Lee, Yufei Guo, et al. Improving image generation with better captions. Computer Science. https:///u.. openai. com/pupers/dallle-3 pdf, 2(3):8, 2023.
[6] Lucas Beyer, Andreas Steiner, André Susano Pinto, Alexander Kolesnikov, Xiao Wang. Daniel Salz, Maxim Neumann, Ibrahim Alabdulmohsin, Michael Tschannen, Emanuele Bugliarello, et al. Paligemma: A versatile 3b vlm for transfer. arXiv preprint arXiv:2407.07722 2024.
[7] Bradley Brown, Jordan Juravsky. Ryan Ehrlich, Ronald Clark, Quoc V Le, Christopher Ré, and Azalia Mirhoseini. Large language monkeys: Scaling inference compute with repeated sampling. arXiv preprint arXiv:2407.21787, 2024.
[8] Souradip Chakraborty, Soumya Suvra Ghosal, Ming Yin, Dinesh Manocha, Mengdi Wang, Amrit Singh Bedi. and Furong Huang. Transfer q-star: Principled decoding for Ilm alignment. Advances in Neural Information Processing Systems. 111101122-1117661 2024.
[9] Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Conghui He, Jiaqi Wang, Feng Zhao, and Dahua Lin. Sharegpt4w: Improving large multi-modal models with hetter captions. In European Conference on Computer Vision, pages 370-387. Springer. 2024.
[10] Lin Chen, Jinsong Li, Xiaoyi Dong. Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang. Yu Qiao, Dahua Lin, et al. Are we on the right way for evaluating large vision-language models? arXiv preprint arXiv:2403. 20330, 2024.
[11] Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing. Muyan Zhong Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation models and aligning for generic visual- linguisic tasks. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 24185 24198, 2024.
10

---

[12] Zhiyang Chen, Yousong Zhu, Yufei Zhan, Zhaowen Li, Chaoyang Zhao, Jingiao Wang. and Ming Tang. Mit- igating hallucination in visual language models with visual supervision. arXiv preprint 1..479311.16449, 2023.
[13] Zhengcong Fei. Partially non-autoregressive image captioning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 1309-1316, 2021.
[14] Tianrui Guan, Fuxiao Liu, Xiyang Wu, Ruiqi Xian, Zongxia Li, Xiaoyu Liu, Xijun Wang, Lichang Chen, Furong Huang, Yaser Yacoob, et al. Hallusionbench: an advanced diagnostic suite for entangled language hallucination and visual illusion in large vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14375-14385, 2024.
[15] Jiaxian Guo, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Boyang Li, Dacheng Tao, and Steven Hoi. From images to textual prompts: Zero-shot visual question answering with frozen large language models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10867-10877, 2023.
[16] Jonathan Krause, Justin Johnson, Ranjay Krishna, and Fei-Fei. A hierarchical approach for generating descriptive image paragraphs. In Proceedings of the IEEE conference on computer vision and paitern recognition, pages 317-325, 2017.
[17] Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference machine learning, pages 19730-19742. PMLR, 2023.
[18] Xirong Li, Chaoxi Xu, Xiaoxu Wang, Weiyu Lan, Zhengxiong Jia, Gang Yang, and Jieping Xu. Coco-en for cross-lingual image tagging, captioning, and retrieval. IEEE Transactions on Multimedia, 2149):2347- 2019.
[19] Hunter Lightman. Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step. In The Twelfth International Conference on Learning Representations, 2023.
[20] Aixin Liu, Bei Feng, Bin Wang, Bingxuan Wang. Bo Liu, Chenggang Zhao, Chengqi Dengr, Chong Ruan, Damai Dai, Daya Guo, et al. Deepseek-v2: A strong, economical, and efficient mixture-ofeexperts language model. arXiv preprint arXiv:2405. 04434, 2024.
[21] Fuxiao Liu, Kevin Lin, Linjie Li, Jianfeng Wang, Yaser Yacoob, and Lijuan Wang. Mitigating hallucination in large multi-modal models via robust instruction tuning. arXiv preprint arXiv:2306. 230.2333..1553. 2023.
[22] Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26296-26306, 2024.
[23] Haotian Liu. Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang. Sheng Shen, and Yong Jae Lee. Llava-next: Improved reasoning, ocr, and world knowledge, January 2024.
[24] Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Lii, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, et al. Mmbench: Is your multi-modal model an all-around player? In European conference computer vision, pages 166-233. Springer. 2024.
[25] Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannanch Hajishirzi, Hao Cheng, Kai-Wei Chang. Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. arXiv preprint arXiv:2310.0222 2023.
[26] Junliang Luo. Tianyu Li, Di Wu, Michael Jenkin, Steve Liu, and Gregory Dudek. Hallucination detection and hallucination mitigation: An investigation. arXiv preprint arXiv:2401.08. 2024.
[27] Luke KyriazKyriazi, Alexander M Rush, and George Han. Training for diversity in image paragraph captioning. In proceedings of the 2018 conference on empirical methods in natural language processing. pages 757-761, 2018.
[28] OpenAL.
Learning to reason with llms.
https://openai. httpe://oopeaai.commineeeee wwwpeenaa..cmmiiiiiaeaeeee index/
learning- to reason- with- 11ms/ 2024. Accessed: 2025-04-30.
[29] Steven J Rennie, Etienne Marcheret. Youssef Mrouch, Jerret Ross, and Vaibhava Goel. Self-critical sequence training for image captioning. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7008- 7024, 2017.
11

---

[30] Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns, Trevor Darrell, and Kate Saenko. Object hallucination in image captioning. arXiv preprint arXiv: 180.00216, 2018.
[31] David Silver, Aja Huang, Chris Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittieser. Ioannis Antonoglou. Veda Panneershelvam, Mare Lanctot, et al. Mastering the game of go with deep neural networks and tree search. nature, 529(7587))484 489, 2016.
[32] Charlie Snell, Jachoon Lee, Kelvin Xu, and Aviral Kumar. Scaling Ilm test-time compute optimally can be more effective than scaling model parameters. arXiv preprint (4di:220000333444 2024.
[33] Yifan Song, Guoyin Wang. Sujian Li, and Bill Yuchen Lin. The good, the bad, and the greedy: Evaluation of Ilms should not ignore non-determinism. arXiv preprint arXiv:2407. 240::24477.4477, 2024.
[34] Zhiqing Sun, Sheng Shen, Shengcao Cao, Haotian Liu, Chunyuan Li, Yikang Shen, Chuang Gan, Liang- Yan Gui, Yu-Xiong Wang. Yiming Yang. et al. Aligning large multimodal models with factually augmented rlhf. arXiv preprint arXiv:2309. 145225,, 2023.
[35] Richard Sutton. Learning to predict by the methads of temporal diffcrences. Machine learning, 3:9-44, 1988.
[36] Feilong Tang, Zile Huang, Chengzhi Liu, Qiang Sun, Harry Yang, and Ser-Nam Lim. Intervening anchor token: Decoding strategy in alleviating hallucinations for milms. In The Thirteenth International Conference on Learning Representations. 2025.
[37] Feilong Tang, Chengzhi Liu, Zhongxing Xu, Ming Hu, Zile Huang, Haochen Xue, Ziyang Chen, Zelin Peng, Zhiwei Yang. Sijin Zhou, et al. Seeing far and clearly: Mitigating hallucinations in milms with attention causal decoding. In Proceedings of the Computer Vision and Pattern Recognition Conference. pages 26147-26155 2025.
[38] Ye Tian, Baolin Peng, Linfeng Song, Lifeng Jin, Dian Yu, Lei Han, Haitao Mi, and Dong Yu. Toward self- colf-improvement of Iims via imagination, searching, and criticizing. Advances in Neural Information Processing Systems, 37:52723 52748, 2024.
[39] Peter Tong, Ellis Brown, Penghao Wu, Sanghyun Woo, Adithya Jairam Vedagiri IYER, Sai Charitha Akula, Shusheng Yang, Jihan Yang. Manoj Middepogu. Ziteng Wang, et al. Cambrian-1: A fully open. vision-centric exploration of multimodal 1lms. Advances in Neural Information Processing Systems, 37:87310- 8736677766668 2024.
[40] Jonathan Uesato, Nate Kushman, Ramana Kumar, Francis Song. Noah Siegel. Lisa Wang, Antonia Creswell, Geoffrey Irving, and Irina Higgins, Solving math word problems with process-and outcome-bass based feedback. arXiv preprint arXiv:2211. 142755, 2022.
[41] David Wan. Mengwen Liu, Kathleen McKeown, Markus Dreyer, and Mohit Bansal. Faithfolness-wwaee decoding strategics for abstractive summarization. arXiv preprint (xXiv:2303.03277 2023.
[42] Ante Wang, Linfeng Song. Ye Tian, Baolin Peng. Dian Yu, Haitao Mi, Jinsong Su, and Dong Yu. Litesearch: Elficacious tree search for I1m. arXiv preprint arXiv:2407.00 00320, 2024.
[43] Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen. Xuejing Liu, Jialin Wang. Wenbin Ge. et al. Qwen2-vl: Enhancing dision-lanuuse model's perception of the world any resolution. arXiv preprint arXiv:2409.12199 2024.
[44] Weihan Wang. Qingsong Lv. Wenmeng Yu, Wenyi Hong, Ji Qi, Yan Wang, Junhui Ji, Zhuoyi Yang. Lei Zhao, Song XiXuan, et al. Cogvlm: Visual expert for pretrained language models. Advances in Neural Information Processing Systems, 37:121475 121499, 2024.
[45] Xintong Wang, Jingheng Pan, Liang Ding, and Chris Biemann. Mitigating hallucinations in large vision- language models with instruction contrastive decoding. arXiv preprint arXiv:2403.1877 2024.
[46] Xiyao Wang, Zhengyuan Yang, Linjie Li. Hongjin Lu, Yuancheng Xu, Chung- -Cing Lin, Kevin Lin, Furong Huang, and Lijuan Wang. Scaling inference-time search with vision value model for improved visual comprehension. arXiv preprint ar arXiv:2412.03770 2024.
[47] Xiyao Wang. Yuhang Zhou, Xiaoyu Liu, Hongjin Lu, Yuancheng Xu, Feihong He, Jaehong Yoon, Taixi Lu, Gedas Bertasius, Mohit Bansal, et al. Mementos: A comprehensive benchmark for multimodal large language model reasoning over image sequences. arXiv preprint arXiv:2401. 10529, 2024.
[48] Zhiyu Wu, Xiaokang Chen, Zizheng Pan, Xingchao Liu. Wen Liu, Damai Dai, Huazuo Gao, Yiyang Ma. Chengyue Wu, Bingxuan Wang, et al. Deepseek-v12. Mixture-of experts vision-languse models for advanced multimodal understanding, arXiv preprint arXiv:2412.1030) 2024.
12

---

[49] Tianyi Xiong. Xiyao Wang, Dong Guo. Qinghao Ye. Haoqi Fan, Quanquan Gu. Heng Huang, and Chunyuan Li. Llava-critic: Learning evaluate multimodal models. arXiv preprint arXiv:2410.02712, 2024.
[50] An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li. Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, et al. Qwen2. 5-math technical report: Toward mathematical expert model via self-improvement. arXiv preprint arXiv:2409. 12122, 2024.
[51] Jiahui Yu, Zirui Wang, Vijay Vasudevan, Legg Yeung, Mojtaba Scyedhosscini, and Yonghui Wu. Coca: Contrastive captioners are image-text foundation models. arXiv preprint arXiv:2205.01911 2022.
[52] Weihao Yu, Zhengyuan Yang, Linjie Li. Jianfeng Wang, Kevin Lin, Zicheng Liu, Xinchao Wang. and Lijuan Wang. Mm-vet: Evaluating large multimodal models for integrated capabilities. arXiv preprint arXiv:2308.028 02490, 2023.
[53] Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, et al. Mmmu: A massive multi-discipline multimodal understanding and reasoning henchmark for expert agi. In Procedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 9556-9567, 2024.
[54] Dan Zhang, Sining Zhoubian, Ziniu Hu, Yisong Yue, Yuxiao Dong, and Jie Tang. Rest-mets Llm self-training via process reward guided tree search. Advances in Neural Information Processing Systems, 37:64735 64772. 2024.
[55] Yiyang Zhou, Zhiyuan Fan, Dongjie Cheng, Sihan Yang, Zhaorun Chen, Chenhang Cui, Xiyao Wang, Yun Li. Linjun Zhang, and Huaxiu Yao. Calibrated self-rewarding vision language models. arXiv preprint arXiv:2405. 2024.
A
Appendix
A.1 Human Evaluation
This section details the human evaluation process used compare captions generated by ViMaR against those from four decoding baselines: VisVM, CLIP-PRM, best-of-N (BoN), and greedy decoding. We conduct blind pairwise comparison study over randomly sampled subset 300 image-prompt pairs from the COCO Train2017 dataset, using detailed prompts from the LLaVA-1500 dataset.
For each comparison, human annotators are shown the image and the two corresponding captions (one from ViMaR and one from a baseline) in random order, without knowing the source model. Annotators rate which caption better using 3- point scale: (baseline is better), (tie), +1 (ViMaR is better). We aggregate these scores and compute the rate
percentage instances where ViMaR rated superior (+1).
As reported in Section 4.J ViMaR preferred in 64.0%, 65.3%, 66.0%, 69.7% comparisons against VisVM, CLIP-PRM. BoN, and greedy decoding, respectively. The detailed win rates are also visualized in Figure [ which summarizes GPT-4o and human preference comparisons across baselines. These results demonstrate the consistent advantages of our two-stage decoding approach in producing more accurate and descriptively rich captions.
A.2 GPT Evaluation
this section, we leverage GPT-4o as an automated judge to compare captions generated by ViMaR against those from baseline decoding strategies. Using the prompt defined above, GPT-4o selects the preferred caption based on richness, accuracy, harmlessness. creativity, and clarity. This large-scale automated evaluation complements human studies and metric-bass analyses by providing consistent, fine-gre tieeggrined judgments caption quality.
13

---

Evaluate the following image captions generated by two vision language models (VLMs) response to a given image.
Criteria for "better" caption:
Richness Content: Provide a comprehensive description objects, actions, colors, and settings.
Accuracy:
Reflect only what is visible without adding incorrect information.
Harmlessness and Appropriatenes:: Avoid harmful, offensive, or unwarranted personal assumptions.
Creativity and Elaboration: Offer imaginative yet accurate claborations that enrich the scene.
Clarity and Coherence: Present a clear, concise, and well-structured description.
After considering these, output exactly one
Responsel is better
Response2 is better Tie
Image: (Insert image here)
Responsel: (Caption from Model A)
Response2: CCaption from Model
A.3 System Configuration and Training Details
All experiments were conducted on a single NVIDIA RTX A6000 GPU with 48 GB of VRAM. We utilized mixed-precision training with fp16 optimize memory usage and computational throughput. The training process was launched using the accelerate framework with gradient checkpointing enabled to reduce memory overhead. The model was fine-tuned on the LLAVA dataset using the provided train and test splits, with a per-device batch size of 16. Training was performed over 4 epochs. The same hardware setup was used to measure inference times for all decoding strategies, including VisVM, CLIP-PRM best-of-N (BoN), greedy decoding, and our proposed ViMaR. All evaluations were conducted under identical conditions and hatch sizes to ensure fair consistent comparison of both efficiency and performance.
A.4 Analysis and Selection of Margin Threshold
To ensure effective reward shaping during value model training. empirically analyze the distribution of CLIP similarity scores across the full training set determine a principled value for the margin threshold Our margin-based penalty mechanism activated when a candidate caption's CLIP similarity score falls below т, enforcing a negative reward proportional to the gap. The choice of 7 directly governs the aggressiveness of this penalty and thus requires careful calibration.
We compute summary statistics over the entire dataset's CLIP similarity scores, resulting in the following: lowest score 0,0031, highest 0.4580, mean 0.2102. We further analyze the distribution quantiles: the 90th percentile (top 10%) is 0.2749, the 80th percentile 0.2544, the 20th percentile is 0.1666, and the 10th percentile is 0.1929. Based on this, we select r 0.16, which approximately corresponds to the lowest 17% of samples in the dataset. This threshold captures a meaningful boundary between well well-grounded and underperforming captions, ensuring that only semantically weak generations receive penalization during training.
This percentile- approach allows to define in data-driven, distribution-aven aware manner, avoiding manual tuning and yielding stable learning signal. By anchoring the penalty trigger the empirical distribution, we promote robustness and generalizability of margin- based reward across diverse datasets and captioning scenarios. The integration of this threshold into our value model's 's training objective is detailed in Section 3.1 where we describe the temporal-difference learning framework and margin-based reward adjustment.
A.5 Temperature Sensitivity Analysis
Our main experiments (Section employ a retterttreee decoding scheme, where ViMaR other methods generate candidates using a diverse set of temperatures 10.1, 0.3, 0.5, 0.7, 0.9). This design promotes candidate diversity, allowing Stage to explore broad descriptive variations and Stage 2 to refine under-grounder segments using samples with different entropy levels.
14

---

Table 3: Comparison of temperature settings for ViMaR. Multi-temperature decoding (f0.1, 0.3, 0.5, 0,7,000000 0.9}) achieves the best balance between caption richness and visual ground- ing, whereas fixed low or high temperatures lead to reduced performance.
Base
Secting
MM--Vel
MMBBand1
NMMUT
MathVista]
CVBaach
LLaVs-W
MSNScr!
CHAIRA
CHAIRU
MMIi rate.
L.LVA-NNss M.MA---No--Muraa--1
ViMaR (T:DA)
46.
521
42.4
70.2
589
21.2
3.W
0.9
VINTER (wutl--rrp))
48K
782
374
415
707
790
393
283
30
ILSI
Table 4: Comparison of reward formulations for ViMaR on LaVA-Next-MMstral Reward' yields slightly better grounding and lower ballucination rates than Reward".
Bane
Setting
MM--Vet
MMBleach!
SIMMU
MarhVann
CVileorh
MMStarT
CHAIRS:
CHAIRI
MMiuaa
(kowanf)
To further analyze the effect of temperature, we compare ViMaR under fixed- tedperatpee settings of = 0.2 and T = 0.6. As shown in Tablc both fixed-temperature configurations Icad to performance degradation relative to the nulli-teemeratuure setup. Specifically, T = 0.2 produces overly conservative captions that lack fine-graines detail, while T = 0.6 increases diversity but also raises hallucination rates. In contrast, multi-temperature approach maintains a balanced trade-off, combining low-entropy high-entropy candidates achieve detailed yet visually grounded captions.
Overall, these results confirm that the proposed multi-tempen temperature strategy is not arbitrary but an essential design choice that enhances candidate diversity and grounding stability. Both the main results (Tables 2 and this ablation study consistently validate its effectiveness.
A.6 Reward Function Analysis
We further examine the impact of different reward formulations on ViMaR's performance under a fixed tempera- ture setting (T = 0.6). Specifically, we compare two variants of the reward function: Reward'. where T. = 8 if & and r., 0 otherwise, and Reward", where T,, max(6, (6, r). As summarized in Table Reward" slightly improves overall caption quality and reduces hallucination compared Reward'. This improvement stems from Reward" providing a smoother penalty near the margin threshold, allowing the value model to better differentiate between marginally grounded and confidently grounded sentences. In contrast, the hard-threshoo Reward' formulation tends to mnder-penalize uncertain cases, leading to minor degradation in grounding accuracy. Overall, these results validate the effectiveness of margin-based reward design and highlight the importance calibrated reward shaping in stable guuueg decoding.
A.7 Additional Case Studies
In this section. we present further qualitative comparisons illustrate the differences between ViMaR-guide decoding and baseline methods. Figures [igure[[throughh] showease series representative examples, highlighting how our approach improves caption fidelity, visual grounding, and descriptive richness across diverse scenes. Additional qualitative results are also provided in the supplementary material to further support our findings.
15

---

Can you elaborate on the elements of the picture provided?
Attention Map after w whole senfence generated
ViMtait Garkled
The lange showes a cusdom living scou with vice of city lay thnouh lange window. The rosu is well-it with satuni ligh, conning in from the winkow Hare are dae key clesaata the piouuc Fanittur: The living joun is Brabledd with a variety senting optans, ischaling sectioal sull, two mandaass a cyliabiicl side takii and rouad orouant The Bunibure appears in of contengaaryy styl with nerina moves fhe corplemeets the nore's docor Fooniig The fivnring kia light cocoed lentwced, which adch wonrth the sgace Windime Treatraett
windins dresud with tee-io-cniing curain that pursally dramn, allencng fora bulance berwoun prozzy and light. Deacr The men
and with petter plants,
wiich louch of gremay und life to the quace Thare ure alao dassatice iacr such books bovd, and vas, witsh cunt to noueth sorielic Lighting: Thare is cclling light stilking fistine soem
fiatue (ial poosidea auclicat lighting roces. cityccape, with trildings
of lhe dlecrali lave winkew. The window ofits u paastattic schene sooan cennzl, with beges, scctntemraai
heights uriting imnoggheee Ceilig: seting Cooe Scheme:: exposed benms chapos
greys
domminng sgace. The crernes
tames,
pervidiu
catteal
cussde.
ViNYM Guided
To image. you tee quctios living room that offers view of city skyline through large floor celling sindow: The noo well-tr with reneaa light coming fhom the wastew, and there is cnling light fistre that procdde addioonal ilbamisstoon The room fomisbed with variery sexting options, includiin plash wits and armdhsiis, all aemanged to crone coerds suble satting ares. The wifa and smnduaio uphoisteved mecral colo pelatic, socod ontoann, which be
complemmanta wamm tmmo the wondan Swning,
aecaiii uxes, thench por decocative plese contr.zees to ronzt's Loonng stnoopoere. comur yamally
coffze
additional sesting the there are coople decountvv zems. for seec comcall
be seriomeee The dryma, reggests ttat rounts ecceptors may wast to conrrol the amorer Light pertacy cocung in orntade Orenil room been tsoegathslly designed create snvie,
only prorude Yeer
CLIP PRM Caideed
The image sboves a spacirus and wej lin living room with a view cityccape thoouh flon co-celing windows mom fiotree a variaty fiminee, inchedings) sectional sots wat baie etahines, which is larget piece Seniturn room circilar ctumnn cenier the soem, which raatches the soda coler achene-3. amachain, of whsch tay with
coffee table fost the sofa, which appean las geouetti pomem ethasce rocni's cocicess. A.A. suodess desiga, wtach rizedd on. adding roonss indoeer lightag a large auta cag under coffee tille, decomtove
poutionnd cleaer the wizdon, providing
caty
decorative leess
porsbbe' scriptrre or plece allowing adjueshie light csennel irvinng chasopler.. cityaspe lackgroua
zom.
poreo place the
hussery, toech preenecy and
speae
glos
image
with laage,
thz spece.
Greedy Deceding
The imags stane madem living mem with viow of'a city deylinc thnuuhh begs windons The nom is with rainn light menngg in windows and thencs o womn sertienne crunted by lighting fistures are amee key sluurata is the picture Fimilre:: The mun is fieiided wils a saricy of opians. inclding seciioall comple of aemchaiis snd seunal otterans fanihure appears of contegoorry syke liass sestrll
Codlee Talle Theshh cedee talle caute coum, soam lirg rig
wilich quee. shope top it.
potted inchiling secall plant coople space.
gramary ganceumic duoo dyylin legs
scalptore
Windowe
frimne with
lighn stacagtere. Lighting trcmnee, Bolcony: Lbeee' vosile wandonss which intereinng archisenuml element to the view. Goerall the rom desipted conrortable and eylist, with feem fonationaliy and serfheicc. The view adda donumic elcaest Bc making a plenanet place rclas and eajos the uebaa stnorodiggs
the
Schene
docilantin gaae
wellaat, with somburrneo
wadowm
Figure 3: Qualitative comparison example I
16

---

Can you describe the main features of this image for me?
Attention Map after whole sentence generated
Emilade
Emirato
VIMaK Ganted
The uuage dloss a sceue Soas socces (Sooball) watch These are three playes the Sasse one weoring Mie jeciey and the othes weaciug a znd Jeney. The playss in the aed lesey, libely from the Arsccal teams is possesion of the yelow zanch bat and grepering ak3k it while the player la the tiue jessy stoupping to kalmoggi or challeage the ball, The payer in werring white usciee and dhamts while player
book goorred seuzi profosimnal reatch. The maaiaan lan analiple scninyy sugesningg bage vanur desggad to asoommeeht: a sqguifkaas smmrer apectaies. The jeneys of player lave spacas logan, well-asinnined,
wrnning Mir ands drons
enoud guectnes inalaating the thic licely a penfosiumel or
conaaoe prolo siesal speats plays soceer sandhes Overal, lange crptrees moneer coupetitve play socce: untcl, tysicel for atmoeghone suggness high-tebbee grmee give interory plyeenan prsence largo
juscs spassus Fly Euiones vialde his jaseyy fir
viabde shatts The gaan
players
image
snough
epecific
VIsVM Grnded
The
treegeened
tie and
trtach
rod white kis wirchh sanecrated with the Assesal footbll clibs, and is a Eeeand puslioa, posillly pessariag seodive ball cake The socoer bull visisle on the groand, andiating that the gasce in The papes the und and whute ann messing yellow abees, which is 2et commooa color Sur soccer dess and might be spenseds color pezional predereace the bociggooud,, thece aze other playes on the fiedd and crond of speconees can sees the caane, siggestig that the a professinal morch benng played dieet of ausieace The grass of the dilt is well. mortiizd, and the sndorn appiors be cundoon. bened oz the rotnrll light and shadows The smnoephere suggers competiive speen event
Ameual football clibs
to defeneeve possill croparin intereent ball paaer hadoommrd
CLIP PRM Gulded
The image cognares Latease zonneer a soccer match There are tiece payers the fiuse, each from ditieent teams. The player the Sureground, werring and jecsey, in actioon. dricking the hell with his telt foot, whilh booking over his danidde the somth-jusyy jawaye here white perenie and the jencey has the sponsee logo tinimine The player in the milbii weading dack bluc junnyy, in deceneeeee stanes banding down and trying anmogee laz ball His icands josays have wlaie yellow arrips un salo, The player is dae beckspoent wostiu u and lemsyy similie to dhe Sest playerl is motion, possilly musiag towand dae bail supporiing his icanuate The crowd in Bhe backrouud a bit of color, indenting large starber of spectatus de sodian. The pruss of the Geld i vilcaal greas bytcal Do soccer pixless The image couveys the coupelitiee spict of the gasse and the ligh level of aldekiiau ou display.
Decading
propering cooed, that waxt condeor The imaae capruee moment with plavr loviin down ponsibly cluning while the player posstooed defend intereapt. the focts os playe and the bail with the crsed servins beckdop the actias Gaté.
programs
ferepreend, player jerney,
ponsenion
beckgooud wilh spectaoor, that professioaal The held,
eppasing peasbbyy
audieace. The white
match siguncsur
Figure 4: Qualitative comparison example 2
17

---

Describe the following image.
Attention Map after whole sentence generated
VIMaR Gulded
onage cones, ioclide ""Cuutioa", "'Electte", " and "Days"", sugzesling that there may be sapeing electrical work oc temperay clos for speciilc dication. The cood lssell nct viuble due the Nockade, and there are no vehieks potestraass is the inmediate viciuity of the coednced-off aucs. The adroukk on the right side of the image shows a fow people welking, and there tow cars and track parbed oc moving os odjicenn lanes. On the lett sice ot the unage, there builing with indieating that is mail salon. The facade the buildngg gartially väibl,, and there i small oundorr seating area with potted plant. The sky clear, and the lighting the
image depicts
sigus with carrion tape.
attached
which are pluced the pavensnt
off These
which appear related
sigus
that roads ("L& Naits",
artificial lights bright
There overpewer amblent light. ovenal people engaging
inage is candid, unposed pantogmaph. capturiag photo
typical daytine.
scene
during
VisVM Gaided
The mage depacts on nrbba street scone with tocms ez section ot the mod that has beet contoned with yellow police tapo. The tope strencted acress the smoct, background,
indicating that is restricted under investigtion. Rehind the Stere several orange tnafiie cones plocod on gevement. further emphaniiing retricted
buikiin
reads "T&G
paked along inmeslani visinll comboncheel evening saggess there may in image. The style the imae is standand, amoeylin fined photngraph that captoros mommnt tufsen
lined people visitle
lamps,
lighting egguses nightt
and the overall stmosphere coninnad off, her moot mamms of incident eavinsment. There are so visible tosta other then the T&G NAILS sigs.
image quiet stil. gresence the police upe
inckdeu that recuired
Cented
trafiic control messuues, The strot condened of yellow caution tape orange concs, indicating that theee may be ongoing consiruction mainvrrance work. beee is visille manlle cover with sign that neads In aattion: Llberrical Work The sign suggets ihoi thice work being done on the city's electrical inft anfrarereture, On the right side of the image, Shee are parked cars, and os the left, thore is commencial building with signage. The dk owercast zad the lighningg suggests it exher lane afteroon or ently evening The evenul otmespbere one au active ubbo enveonment with ongoing city mountenance or consirusion activiliees
image dapicts
steect sease with
Greedy Decoding
The image depicts street scene with focus az section of rood that has been coedoned off with yellow police tape. The tape is attached to several omane traffic cones, which use placed in way taat blodia the conkord-off area. The tape and cotes cleady viable and tppeur be place notrict socea the area. la the backgooud, snectiigns,
"TBG NAILS brilding evercast, sagpestigg migtit cloudy day: The style image standard nos-arylzed phonograph copmires real-ite scene.
commeccial
stocetion: with dispay plants. of busnesses
the
sky
Figure 5: Qualitative comparison example 3
18

---

Attention Map after whole
What's happening in the scene?
sentence generated
VIMAR Gulded
ibere clock with clocks visible lis facade, The day overtoal with dark clouds, suggeniing might cluudy clock
Lse fongrrondd but cleasy public space, poesibly park activity There sering appeans
brildings, indientnng moving bockground, indicatng some geven przsonce clock tower and the hencites visible foregronad. people in the immediate vocinity of the tower The image captnces quie moment in whar might be busting area during otter times of the day. he presence the fing on top the lower adds a touch of aational pride or identity the sccai. There are power lines running accoss the image. The ovenill composition of the image, with the clock tower the lical point, draws the viewer's eye upwagd empboszrng the beight and prominece the sowes the bundsauee. The image is snapsbot of momment time, captiriin the stillaesa of the scene degitt the potestial for activiyy the backgooudd.
the inssediste vicaiity
squaee, due the ovcocace sky and choence
image is
VIVY Gended
cleady visible. agpears cloudy, suggesting parked slong presenee flag un top of tower indieties might public municipl building, posalily town hall similar instittion ovenall simosphes: the inage sommwhat somber the clendy sky and atonnce of people in immedooe vicinity tower.
forogoound
sppnosching: background,
buikiings, ancluding prominett
goven roof, street
CLIP PRM Gatded
image
top, might
governmen hiilding
suggoeing might cleudy
Relew the there's strzor scone with shops car, indicatin lilely city center. The fleg ilyngg halt-mest sigutyy period mouming
special evenr that requires the slag to be lowered.
Greedy thene
appears prominent landmark, poesibly
city
evercest
located area electiioal infisiructure Soreground, there are sume uees and wazi locks like a public space with beaches, wtichh could be a park a simiar peceatiinaal area. There are also socne bulldings the background, which might be part of comnicciial reaidential auos, The ovenal umospteee of the image cain and quitt, with se visible setivity that would indicate a specific event or tite et dey.
suggesting night cloudy day, There are power rusning across image, adicating
Figure 6: Qualitative comparison example 4
19

---

Attention Map after whole sentence generated
What are the key elements in this picture?
YIMaR Guided
the warms tlanc is red douhilddscker bus driving down strat what appeoers to be city oenter. The lus los sigs on Sae frrst Bai nokls THHE WEDDING SPECIAL, and ("wwn nimebas co tik." and License plate "MD 3est3* indicntng that it likely being used for weeking event There flecal amangement on the finat of the bos, which a common daccration for wedklings. in the backgooudd there are podlstriens and other valicles suggosing buy urhan envirenment. The sechilectire of the The
obtea für sightseeing and cebehmior visible elenzents image and inclide any specolative or indered intormation. imaze poovide enough context to determine the exact Jecation the spectiic natrre of the wedding event. The bus could be port ot four compray that otfers wedding specials, or could a porately lired bus for specitic wodding.
buildings and the style bus
that this could be
London, where double deckae
seent overcaas, Please nose
capture momeett everyday
bis's signng
descciptios
VIeVML Groded
the tmditicual red duuble-decker driving stree,, with word "WEDDING" waitten it frust, indicating people walking sidemalis, and
be used for wedding
flonal crmuggenenn
which nsed such vehicles special classic archneectual siggeeting scene likely is city strong Hiittsh each The presence as pedeariaus, andieates typical utban euviromment. weather appears be overeast, and there inzmedlaee sign of any
trilkiing
catlic Lighs other vehicles, specillc event beyond the fact that the bus being used for wedding.
CLIP-PRM Galded
the scene, there is ned double decher bus driving drwnn city street. The bis has sign on the front that reads ("T WEDDING SPECIAL and "wraee timerable co ok.." whichh that the bus being used for wedding. posulity unique made of tttuspporation the wolding party or guests, The bus decumned with flowes. which urban with velicles pedectians.
weather practice
Thece wilking the sidewall, and the appears
indicating
spring
Greedy Decoding
simelesbus okl"" incicoting bus specitioally rsed foe wedding tnasportztoz. The decocated flowen, waich common peactice wedding vetucles destive touch. There are people wilking en the sicewalls, and street appeors to be in city wat boilings and tritic lights visible in the bockground, The wenther seems be oveecast, and there are no visible sigas of rain or stom. The eve atmonghhere suggests opicall day az wiee settung.
driving
sigs
WEDDING SPECIAL!
Figure Qualitative comparison example 5
20

---

Attention Map after whole sentence generated
Describe the following image.
VIMaR Gasded
The image depicts a serene scone of group of people nt boat in body of water, which appears to be a river or lake. The bost is long, darmo vessel, with pointed stom, and it's being poled through the waler by indiviiall standing the zeor. There are sevecal people seated in the boat, all facing the samee direction, which is upwards ibe lell side the image, saggusinng lnssurety actisry, possibly sighnseeing sanry day backgoand, thene consimeted stits, indicating that location may peone flooding msefs, which pertillly regoo where constriction common prrsection against risng levels.
moving spaream towands the lift bank. The bost is gostioned the feeggrondd with the peogle oabosed appering be ergaped standng
tour. Tey dressed
and sugzesting might buis, and the overall color puletle the image dominored shades of grven, with
prsmury
The
water has gemile ripples, reflectnng the grvrney controst with their earts lones. The image has watemark a Elter appledd to giving it vizage or aged appeozance, with a cliati sepua tose that eabances the zatund color: and adde a musnigii fiel the soate. Thrre visible unci the image, and the style of the image saggeds il could be photogragh takon by a lorcist a taveler. captiring a momest daring thoir yoar to coltiral natural sme. The compestioo the inige, wit the bect in the toregronnd and the zuts the hocagrecad, lends the stowerrs from the host
and people prov
cocoparns the trnquil pichereque ertiing beyend them
VISYM Guided
The imags degicts tranquil scena on body of water, which appeers to a lake or sleg-mmming river. In the Breground, there as wenden bost with several people on boand, They seem be enjoying a lesurely ride, with one person standing at the froat of the best, posably steering or nevigningg The best painte in reddici-brenn boe and low tralitional doign, engnnling the water's edge. graan, inficaing nopical subtroical climste
be used fir toespeataaion komiam segimn where such losts cammunss Th backgunndd flas several flanchaal- local communiy tonrie faciliyy the surounding which
hrrs bungslows ritured vegetation
sirictres have motic
and are likely
stmospe the of photograph natrralisoc, cppinting sctne wahout apparezt filees
graesary
tranling the image, and maipulations. The overlll mood the image is pescefil and idyllic, eooking a semar of rebesation and iecape from the busile and buslle sveryday Esr.
scene There no
PRM Guided
the image, small woodes gently floating on cale, gresnsh browa rier. The bost,
with people who are pointed froat slightty curved ofles, suggeeing a seree
enjoying a day ond camanderie and shared eajovmest. The river msetf is nested mmddst hush greeuery, wiS sevecal buts hit-lkke strectures visble on the store, These strucmues, aicey made of bombon, have itaiched roofs, blending hermonoondy wis the natural suroundings. he vegettion appear danss, craiing s senme and secluded envirnment The beat is moving coay fiom the camena. gring view is cear ventues furter donn the river. The pesspctrre from the shore looking somards the hecr creates sense of image. evenall
water poosenges enosoal cassed cinting wone dunnnng lats for stalle They are sitting clase
deptt distanoe
one leisure and canquility capturing moment relossioon enjoyment in natural wrllnn
Grerdy Decading
The image depicts settne scete of group people a boat in body of water The boat is madiional design, poesibiy a cance a small monorboot, and is occupted by several inlivadual who sppear be esjoying leisuelly outing The basi moving away foo the shore. which is lined with dunchedd-coo lusss suggeslig tropiaal or the sunousding vegetztion, which incoudes
rubuopical locaioon The calm, rellecing the the peseeful alyliic eapturing
suanos. reletation enjoyment
littt
The overall
Figure 8: Qualitative comparison example 6
21

---

Attention Map after whole sentence generated
What this photo about?
VIMaR Gadded
young shonry,
graay trees, siggesting
appeaes
passonal setting. posably right prodeminently
black markings. background, The jambs reem be gracing esgloring the Teld, which is typical behavier for young sheep they leeza to uavigate their eovinumen. presence other sheap in lhe socktaround iodicates that this past a ligger dlock. The photo captizes a peaceftl and natural moment is the lives ot these youns aninals. The weather aopears be ovences, and the grees libh and green, indicating might be spring summer when trees ace not fniyy Jestid. The evenll scane conveys sense of traspility and the zample becuty maal landscapes
sheep,
where sheep
light-coload strpe walking in gress Seld, are backinrand, including what appeors ducks groring snimals tras hare brunches, which sugpesos it might sgring early summer ben trees noi fully leated, The preseace of both sheep and ducks andoones mixed mixed-species covinameut, posailly on a diam or in runé sctting where differest types of livestook can be lound togetherr The same penssffll repesentee a rypied day in mial ans where animels ver free to nom and grave
ghoto depicts pastoral scone
young shoop, black
CL.IP FRM Guided
sheep, one natural sontin suggeats this or agricultzall The sheep appzer caln likely part of that duily the tame: The photo cepmoee a poscotul moment in the lives thes anmais, shomcasung their netural behomnor in.8 pasture.
pboto degucts
while, togeierr grazy
aze sheep bydkazrudd aracing resting.
presesie
Greedy Decoding
Welsh beouazin, lrabb known bembs, fied. bmb lell
be photo shows
blok faced possitly breed like Jacub Black black-taced
back whats could be beced Like Black Welsh Mouunaia They togetber, backgreund, there sheep trees, Figgesting pestocal seming ponsenoe birds, possibly ducks, pasteral scene.
Figure 9: Qualitative comparison example 7
22

---

Can you describe the main features of this image for me?
Attention Map after whole
sentence generated
VIMaR Galded
the imagn, there are two children sonding on sidewalk nest a white fire hydrans The child en the left is wesring red reventer, bise jeens, and white shoes, while the child on the right is dressed a dack blue swessshin with hood, blue-wbite pasts, and white shoes, Both childres are smillng us the camena appear to be posing the photoe the bocigrwund,
parked
briiding with ficade The distactive vauble
envinommen possibly residniial
the aydract
buildings. The sky clear, and lerdmar,
ovenall
cheertul 1"S
backgronad servng content which sypically used idensify hydront firefighting purposes. image well-it sezms been talion during the day under ecnursl light. composation the panto belanced, with tie childres contrally pecitinned and the dus Iydiant their side.
connal. tocus
chuitres
toydrant
VIsVM Guided
In the there chiltrrn steding sidowalk nesd white fire bydoers The child on the lefl weering red ewenter and like jomrs, and the child the righe white steokees. They both appear be standing residential there buikdings the sunding
dressed dack bise boodie with white drwwstnngs, blue backgrand and neaeby. porson
sidewall, there car, and the distance, thore ace more vehicles and buiddings The chilcoea seem posing for the photo, with the right looking bu more sobemn than the other, The evenil sceae casual and en evydday, caplucing a moczeni of chilchood in an urbez sellitg.
some goried
fietber
sidewcils, right image,
CLIP PRM Gatded
this imags there ane twon youngg childhen standing on sidowslk The child en the left wosring not mwenter and pair white shows. They lovee short lavir and ane smmiling the camen. The child on the right wearing a dark blne ovestiiin with red stripe the side, paized with dack pants and whine sneakers, They have short, curly hair and abo be smiling thet the two children, there is white fire bycon., The hystemi his chain stoeceed ii, ibene sumbers and lotters visible siden, sidewnik. lighting
abtioual: specitic details
The stany day, and are people other
typical
buildings, viciait.
childres in
Greedy Decodinng
In the imags them ane twwn childrn stendin on sidkwalk noad a white fire bydhant. The child on the left is wearing a swenter and blu jomns while the child on the right is dressed in bize jacket, bloe parns, and white soeakers. Both children are smiling and appear be posing for the photo. In the backgronnd there are parked cars and brilling with nel Guzade the suggess urban envintmeni, possibly nesdemial area. The fire to-trai poominet fesure Goreroundd indicating like
Figure 10: Qualitative comparison example 8
23

---

What do you see
Attention Map after whole sentence generated
happening in this image?
B le
B
le
BNP PA 8 AAS
BNP BAS
OT
OT
VIMaR Guided
In lee tlare is male tamis player the miidiie of serve on clay couet Tee wercing light bluz slin,, dark the dunnt, and white alaces The player hoidling aed white racket and spactaaions watching
striking yellow tennis bell, appeors focused the ball, which mid-nir the background there crowd sperounnd by campany. The suggrating indicate be is executing powerisl serve, which is crucial part of the game tenuis. The chty court surfice is chumcceistic of the Tranch Open, one of the four Geadd slam temis loumaneais. The specialoes are seated on chzirs, and there are potte plants and flowers adding to the ambiance of the event. The evenil scene capaures the inerrayy and excrement of live zennis match.
them groen advertisement
with
setting suggests that this professicnal high-level amaters vetnis match player's pouture the position the
VWVM Guided
sae image, thece teune player the middle ot serve. The plarer weecise shut and shors, with whute white froms yellow hull other hand, proparing to
teuns meiset red and
The player looking upwards lisely tracking the halfs rajestery In bockground there's a green adverssemen boaed with dhe tor I"BN PARIBAS? and a star lopt, which supgers that the event being sponsered by BNP Partbs, a French mullins/ Soral banking corperation. There are also spectators is the stands watching the mutch The willing appean. to be a prsiiiinnal tamis cour.
air
CLIP PRM Ganded
Euage, the is werring
TIe the mibile of's leeckland slas, with nacke positionaal Iediial his sumaal aruaads unde playe: attins shoes. The player appenss fecused ball, which
him. background, are spectsiom woiching the motch,, and these is sign that roads MTNN BA188 which i lirely a spunser evnnt The watting suggests this is prrfrssioral iranis match.
shorts,
Greedy Decoding
the there male player middle soing, hitting bell
appears middle mucc, indicored the while which typical
behinal The playe worinng siin, teunis player. The sotting suggests thi prodessional match.
croud background
adher
Figure 11: Qualitative comparison example 9
24

---

Can you elaborate on the elements of the picture provided?
Attention Map after whole sentence generated
SHANNe OPEN
SHDRAA GHire OPEN
VIMAR Guided
The image shows beach with uevend ebements Surbbourth There four sucdbourds leening agrinsi millng the side of image They of various colors sugaesting wilking They appear enjoying difleeest indivituals Beacchooess: There
strading the hockground, you can 920 large cliff headlead jumtig out into oceen, adding a magedd naturl tecmure beckground, landscope. Roiling black
ervironmen. Oceen:
expanse
beeaking
railing niss aloog the edge of the beach, prowiding a boundary between the sand and the rond Signage: there is sgn on the right sude of the image that reads ("SAL sat, RY SURE SHOP OPEN." indicatnng that the surf shop open for business. 7. sky. The sky is clear and blus, suggesting a sunny day, which ideal for heach activities Road: The road runs panillel the beach, 2nd appears be public road with a sxeewalk. The evenil ammosphere of the image relastd and leisurely cypical of day at the beach.
Cuided
The image depicts lively beach somne un clese day with bire sky and visitle clouds. The beach a sandy and appears to be a populer spot, zs thece are sevenl people scamtordd across the areo, some of manm are siming on the sand, while others are sanding or wallingg In the toroground there is black metal miilng that sopartees the beach from paved ares, wiach couti a boardwalk or paza for pecestnans. Leaning against the millng suthhoads vareess coloes, inchiding yellow, lito, and wttrto, sapgosting that this is place where surfing is popular cctyvity. On the goved ares. there is sigs that reads TSALTBUUR SURE SHOP OPEN indicating that there shop suggests
purchase surling equipment. might
the surbboards and
beach known sucting appeses be calas geuzlle
pooibly allas suats. beacs linel graaay The emal rmmoppece of image is ose a recueational cnvironment wheee people come enjoy the beach water actvvitles ottecs.
picticking jusi sclasing,
CLIP PRM Galded
The image depects scenic beach veew: In the toregrornd, ther''s a menl miing along the edge ot the beach, like to pervent people from accatentally talling unto the oceaz delineate the boundary of the beach area. To the right of the ralling Siore are severa surfboards leaing aquinsts sign that *RAI. TBU SURE SHOP OPEN. indieaing that shop
open people eajoying day.
for sale vishors. Beyand small teessnes Boo beach. occan viuble backsroud, with waves indicniigg
The surfboards
didferent colors and
suggestig spockled Eongrists
leach apyeens be quite bay with
luckeie
whach othen used collet sbells and hile, saggeting gool wrather. scone and is chanieeriate Stature of the conatine: Oveull, the image captures typicaa day ar beachaid: aus shop, with people eajoying the besch and the shop offering serfing equimment for hine or sale.
sea, sky for backgrand, thend' bege firmttion that estends imt the oame. This eliffiike strchor adds doeraaii buchiin the
Deceding
semue with Adluwing eleneats: I. Sufllaad These sevend lboseds laning syaine milling the lell side the inage surfboards vary in color and desigs, suggenting they beiong different indivicuals The presence of the surfboards indicness that suring is popultz activity this beach 2. People and hoach activity: Thure ai severa people on the eealy houch Sume see closr in the water's edge whil: oflers are tirthe back, proaibly santsching savisliving There are also beach unbreelae set up, providin shade for benchggeer. The presence of people and umbeellas suggests that it's smny day and the beach is poprine spoc for néccation and leinure activtlive. Cleenn: The body waver visile, with waves emshing agains the durs, whish common feeturs of coostal beach. The ocentrs Clill the background,
lee imags you can sce
the brokaape.
clill beadinud juttnn
coen, nigged
Figure 12: Qualitative comparison example 10
25

---

Attention Map after whole sentence generated
What do you see happening in this image?
LIGHT
LHGHT
ODEO FANS
ODEO FANS
VIMaR Guided
la the mmge. there person riding bocking bronco dunng what appears rodeo event The nice is weening contooy and ummeerd the bock, whatt oypical attre for a rodao comgetiter. The hoese in mid-back, with it and raised off ground and the rider is holding onso The hoeae's roins, stying In the background, there
watching
words "LIGHT RODEO FANS
that this part rodeo las inslistt
selling lnoks die which
vodao compettiions. The rala's postuc
rocuired spoct. spectators focused contonvenial
event. The izage capezzes moment high sdrenaline action, andeaiing die excileneeut and lnteuiity animals involved, porticulsely concarss
sibletic prowiss. Pleass nota that molion events hacktigg The inage does not proside enorgh contett to detemnine the specitic event or location, but is clear thet this is professioual indec event. The rider's amtr and the presece of membored veni sugges (zat this is congetiive event, and the rider is lisely a profensional condbny..
the walfire
VIsVM Guided
riding hoose, likely paricipating
and the that includes
and seating. The rider
hull-Jugging competilinn, the godl theww suga the logs leenicg foowand, typucal fox nntemptin casch steer. spectiorss wancing gallop, LICHT RODEO FANS)" suggesing that this a.a publicc event where midio aguets see pecsent. The setting looks cypical rodon arema, with borniess for sabety.
last appcars be eagagei activity
mni--
is bemner tackground
CL.IP FRM Guided
the image, Litte rodec evezt taking place A comboy is riding a bocking benaco, which hone tnined to buck and throw the rider ofl back. The cowboy helding outo rigging which is strap that staches to the horse's halter and the sider's belt, allowing the sicer stay the hoose. The horse is mid annls, and the comboy is leaning balance. andience watching and atmnoghore
forwzed The sutting apuears
there competiivee
backppound that "1 RODEO FANIS"" indicaning that this
Greedy Decoding
the imagr, there mides event taking place comilvy is riding a backing hone, which is common evert in reden comptttions The monhey holding onno a rigzing, which a stng that helps him suy ou the hoese The bosse is mid buck, with iss hind legs rised off the ground, which is typical behavioe for bucking hoose duzlng zodeo combuyy bai and vesl, which typical alline a todeo cun boy. sudbence watching evenl, and banner the bockground appears
The cowbooy
"LDGGET RODEO FASS?" inducating
spectators.
Figure 13: Qualitative comparison example 11
26

---

Attention Map after whole sentence generated
What are the key elements in this picture?
VIMaR Gunded
this picture buncketa. fraabe neon green appears
perforning biik thee relling
pesson wearing blue, suggoating good wasther cenditions. A
white sports and bodkgroond,
beach
The beckgoaand inclades while pornaols and people selacing. indiasing recrestional ammoghere. blue and yelow mot lasc the stad. S. there is bost and some beach gear in the bockggoond, renfocting the casusl, constal setting 9. few indrvsdunls nee suttngg lounging the sand, 10. The overll composioon of the image conveys sanse of energy and fin, cxpturing a momert of atbleicam and vejoymers.
VisVM Guided
elements this poeture
mid perfoeming
thibee. The frisbe yellow The wearing red and black lesel. backguuadd shaww sctting with beach
white shoss with black The persm lonalnse beachgoers. The sky cleas, suggoatin good weather conditiuss There body of wateer visible distane.
the image, saveral key alemente I.A, panson mid-ein, perfeeming frishee trick. They are worring shin black shors, they thei bead, The fribbe is air, close the peroetts feet, indictting that they are elizer about cauda it or have just thowg 2. 3. The person is on beach, as evidenced by the sandy ground and the prnsence of beach umbrellas and chairs the bwikgroind 4. there aee other people en the beach, some of whom sesied under umbrellas, supgesting relixtdd Jestuelly ammospbece. The say clear and blue. indicating good weather conditioss The evenil scane suggests recrestional activity. possiiiy dry beach.
fondara
cacual
taking
Greedy Deceding
tais giemas, Bere are several key alemens:: A person mid-eit, parforning a frishss trick. The porsen is wearing a rod abirt and black shors, and they are holding a yellow trisben sandy beach wettng with a clear blue sky, A.A group of people sitting under umbeelles, which siggests leisurely beach enviroment. 4. bast vtuible in the backgouud, inliaaiin that the leads might be near lody waie. A blue and yellow of beach eqripment The dynomic and friehee monien, which oxpture action skill involved mick The close sky and the hright" sunlight, which comiibut
sand which could be the sctup the fiilus uick simply
energtic atmosphore
Figure 14: Qualitative comparison example 12
27

---

What do you think is going in this snapshot?
Attention Map after whole sentence generated
image, sas a persan eseraling a gaved seca of ding with closical anditectur: The persmn hulding a fatnis game or practice sersion, as indictted by the yellow- green tennis hall in motion nearty. The persos unopnely dressed what sppeaes be a whire mobe toga like cunift wish red crosaes and blue sccents, possibly costumee or taemed iee milher that typical gpottewear the backgroumd, the selling ippaa to be ouside a clansical-sty -ttyl building with till colomue and sione walls, revembling hisonccl or gevenmentall scucture, likely crcuni laacmark Seveeal people aathered behind black millng the platfomm early
appraes
misblic
that for low the skay which could indicane shndows the ground actrmnie late afternecn. everall stmosphere relased and lesurrty, people might suggest they bronght personii belongings acditional spects equimeea with them
buikinng observing
below,
toclspack
person
VisVM Guided
micket and appears middle are several other people backgronud, some whem are sandlnn on belcory above the tennis player, possibly weaching the game prictio The setting appears to be urban envineemeni, possbly's public parks or rec cationaia crea within city. The presence of the tanris ployer and the sport aquigment suggeas that this is an aunoor recrestional space where people can engage in sports activilies
image, person
of building with clasicall architeture, individual bolding mosely, sugesting
nennis
sponts
perann might
pesson diessed sports atture inchides white outti comumendy associated
CLIP FRM Guided
stancing paved bolding vennis wearing while outii with red and buse indictted presenet of the uodmvdmnls the sazte
teaais
sugaestigg
The persen appears trilding cohmnne, whem are standing on balcory or raised plaffeem poscibly warching person play tannis, The setting soems outdoons and the ge presence of tennis hall macket sapgeas thet the person is engagedd in sporting activity.
middle a pubaic traiding or sports fncilry. There
anc bockgroend,
Greedy
sugaests it mighi a pubái instiuicual building. penson bolding tenals stcket appeess to be looking at teusi ball the goond. It secnus like the persou night be takingg bueak playing other people in the bockground, some at when are standnn oz balcomy, which undicates that this might be popolar spot for people to goster and socustize. The preseace ot bockpack and hundhag suggest that the per son might be suadent or a victorr to the ares. The overall comoophere is cocual and relexed.
de image,
person stmnling en paved
brilding
guane are
Figure 15: Qualitative comparison example 13
28

---

Can you elaborate on the elements of the picture provided?
Attention Map after w whole senfence generated
ViMtait Garkled
The lange showe a cusdom living scou with vice of city lay thnouh lange window. The rosu is well-it with satuni ligh, conning in from the winkow Hare are dae key clesaata the piouuc Fanittur: The living joun is Brabledd with a variety senting optans, ischaling sectioal sufl, two mandaass a cyliabiicl side takii and rouad orouant The Bunibure appears in of contengaaryy styl with nerina moves fhe corplemeets the nore's docor Fooniig The fivnring kia light cocced lentwced, which adch wonrth the sgace Windime Treatraett
windins dresud with tee-io-cniing curain that pursally dramn, allencng fora bulance berwoun prozzy and light. Deacr The men
and with petter plants,
wiich louch of gremay und life to the quace Thare ure alao dassatice iacr such books bovd, and vas, witsh cunt to noueth sorielic. Lighting: Thare is cclling light stilking fistine soem
fiatue (ial poosidea auclicat lighting roces. cityscape, with trildings
of lhe dlecrali lave winkew. The windoe ofits u paastattic schene sooan cennzl, with beges, scctntemraai
heights ursting imnoggheee Ceiiig: seting Cooe Scheme:: exposed beams chapos covioraced cussde.
greys
comminng sgace. The crernes
tames,
pervidiu
catteal
ViNYM Guided
To image. you tee quctios living room that offers view of city skyline through large floor celling sindow: The noo well-tr with reneaa light coming fhom the wastew, and there is cnling light fistre that procdde addioonal ilbamisstoon The room fomisbed with variery sexting optionss includiin plash wits and armdhsiis, all aemanged to crene coerds suble satting areu. The wifa and smnduaio uphoistered mecral colo pelatic, socod ontoann, which be
complemmanta wamm tmmo the wondan Swning, foosk aecaiii uxes, thench por decocative plese
coffze
additional sesting the floos there are coople decountvv zems. for seec comcall light. contr.zees to ronzt's Loonng stnoopoere. comur yamally
be seripmeee The dryma, reggests ttat rounts ecceptors may wast to conrrol the amorer Light pertacy cocung in orntade Orenil room been tsoegathslly designed create snvie,
only prorude Yeer
CLIP PRM Chideed
The image sboves a spacirus and wej lin living room with a view cityccape thoouh flon co-celing windows moo fiotree a variaty fiminee, inchedings) sectional sots wat baie etahines, which is larget piece Seniturn room circilar ctummn cenier the soem, which raatches the soda coler achene-3. amachain, ene of whsch tay with
poutionnd cleaer the wizdon, providing
caty
coffee table fost the sofa, which appean
decorative leess
suodess desiga, wtach rizedd on. adding roons's indoeer lightag a large auta cag under coffee tille, waxch las geouetti pomem ethases rocni's cocicess. A.A. decomtove
porsbbe' scriptrre or plece allowing adjueshie light csennel irvinng chasopler.. cityaspe lackgroua
zom.
poreo place the
hussery, toech preenecy and
speae
glos
image
with laage,
faz spece.
Greedy Decoding
The imags strne madem living mem with viow of'a city deylinc thnuugh begs windons The nom is with rainn light mennng in windows and thencs o womn sertienne crunted by lighting fistures are aaee key sluueata is the picture Fimilre:: The mun is fieiided wils a sarict of opiana. inclding sectioall comple of aemnhaiis, snd seunal otterann fanihure appear of contegoorry syke liass sestral
Codlee Talle Theshh cedee talle caute coum, soam lirg rig
wilich quee. shope top it.
potted inchiling secall plant coople space.
gramary ganseumic duon dlylin legs
scalpture
Windows
frimne with
lighn stacaphere. Lighting trcmnee, Bolcony: Lbeee' vosile wandonss which intereiing archisenuml element to the view. Goerall the rom denipted conrortable and eylist, with fexm fonationaliy and serfeeics. The view adda donumic elcaest Bc making a pleaanet place ralax and eajos the urbaa stnorodiggs
the
Schene
docilantin gaae
wellaat, with somburrneo
wadowm
Figure 16: Qualitative comparison example 14
29

---

Can you describe the main features of this image for me?
Attention Map after whole sentence generated
Emilade
Emirato
ViMaK Ganted
The uuage dloss a sceue Soas socces (Sooball) watch These are three playes the Sasse one weoring Mie jeciey and the othes weaciag a znd Jeney. The playss in the aed lesey, libely from the Arsccal teams is possesion of the yelow zanch bat and prepering ak3k it while the player la the tine jessy stoupping to kalmoggi or challeage the ball, The payer in
werring white uscice and dhamts while player
wrnning Mir ands doons
book goorred ihem enoud guectnes inalaating the thic licely a penfosiumel or
seuzi profosimnal reatch. The maaiaan lan analiple scninyy sugesningg bage vanur desggad to asoommeeht: a sqguifaaas smmrer apectaies. The jeneys of the player lave spacas logan, well-asinnined,
conaaoe prolo siesal speats plays soceer sandhes Overal, lange crptrees moneer coupetitve play socce: untcl, tysicel for atmoeghone suggness high-tebbee grmee give interory plyernan prsence largo
juscs spassus Fly Euiones vialde his jaseyy fir unmber viabde shatt, The gaan
players
image
snough
epecific
VIsVM Grnded
The
treegeened
tie and
trtach
rod white kis wirchh savecrated with the Assesal footbll clibs, and is a Eeeand puslioa, posilly pessariag seodive ball cake The socoer bull visisle on the groand, andiating that the gasce in The papes the und and whute ann messing yellow abees, which is 2et commoo color Sur soccer dess and might be spenseds color pezional predereace the bociggooud,, thece aze other playes on the fiedd and crond of speconee can sees the caane, siggestig that the a professinal morch benng played dieet of ausieace The grass of the dilt is well. mortiizd, and the sndorn appiors be cundoon. bened oz the rotnrll light and shadows The smnoephere suggers competiive spoen event
Ameual football clibs
to defeneeve possill croparin intereent ball paaer hadoommrd
CLIP PRM Gulded
The image cognares an Latease zonneer a soccer match There are tiece payers the fimse, each from ditieent teams. The player the Sureground, werring and jecsey, in actioon. dricking the hell with his telt foot, whilh booking over his danidde the somth-jusyy jawaye here white pererie and the jencey has the sponsee logo tinimine The player in the mibbii weading dack bluc junnyy, in deceneeeee stanes banding doon and trying to anmogee laz ball His icands josays have wlaie yellow abripe un salo, The player is dae bockspoenn wostiu u and lemsyy similie to dhe Sest playerl is motion, possilly musiag towand dae bail supporiing his icanuate The crovd in Bhe backrouud a bit of color, indenting large starber of spectatus de sodian. The gruss of the Geld i vilcaal greas bytcal Do soccer pixless The image couveys the coupelitiee spict of the gasse and the ligh level of aldekiiau ou display.
Decading
propering cooed, that waxt condeor The image caprree moment with plave looiin down ponsibly cluning while the player posstooed defend intereapt. the focts os playe and the bail with the crsed servins beckdop the acikas Gaté.
programs
ferepreend,
player jerney, that professioaal The held,
ponsenion
beckgooud wilh eppasing
audicace. The white
match siguncsur
Figure 17: Qualitative comparison example 15
30

---

Describe the following image.
Attention Map after whole sentence generated
VIMaR Gulded
onage cones, ioclide ""Cuutioa", "'Electte", " and "Days"", sugzesling that there may be sapeing electrical work oc temperay clos for speciilc dication. The cood lssell nct viuble due the Nockade, and there are no vehieks potestraass is the inmediate viciuity of the coednced-off aucs. The adroukk on the right side of the image shows a fow people welking, and there tow cars and track parbed moving os odjicenn lanes. On the lett sice et the unage, there builing with indieating that is mail salon. The facade the buildngg gartially väibl,, and there i small oundorr seating are with potted plant. The sky clear, and the lighting the
image depicts
sigus with carrion tape.
attached
which are pluced the pavenent
off These
which appear related
sigus
that roads ("L&G Naits",
artificial lights bright
There overpewer amblent light. ovenal people engaging
inage is candid, unposed pantogmaph. capturiag photo
typical daytine.
scene
during
VisVM Gaided
The mage depacts on nrbba street scone with tocms ez section the mod that has beet contoned with yellow police tapo. The tope strencted acress the smoct, background,
indicating that is restricted under investigtion. Rehind the Stere several orange tnafiie cones plocod on gevement. further emphaniiing retricted
buikiig
reads "T&G
paked along inmeslani visinly comboncheel evening saggess there may in image. The style the imae is standand, amoeylin fired photngraph that captoros mommntt tufsen
lined people visitle
lamps,
lighting eeguses nightt
and the overall stmosphere coninned off, her moot mamms of incident eavinsment. There are so visible tosta other then the T&G NAILS sigs.
image quiet stil. gresence the police upe
inckdeu that recuired
Cented
trafiic control messuues, The stoot condened of yellow caution tape orange concs, indicating that theee may be ongoing consiruction mainvrrance work. beee is visille manloe cover with sign that neads In aCttion: Llberrical Work The sign suggets iboi thice work being done on the city's electrical inft anfrarereture, On the right side of the image, Shee are parked cars, and os the left, thore is commencial building with signage. The dy owercast zad the lighningg suggests it exher lane afteroon or ently evening The evenul otmespbere one au active ubbo enveonment with ongoing city mountenance or consirssion aetiviliees
image dapicts
steect sease with
Greedy Decoding
The image depicts street scene with focus nz section of rood that has been coedoned off with yellow police tape. The tape is attached to several omane traffic cones, which use placed in way taat blodia the conkord-off area. The tape and cotes cleady viable and tppeur be place netriet socea the area. la the backgooud, "TBG NAILS brilding evercast, sagpestigg mugtin cloudy day: style image standard nos-drylzed phonograph copmires real-ite scene.
commeccial
stocefion: with dispay plants. of busnesses
the
sneetligns,
sky
Figure 18: Qualitative comparison example 16
31

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The abstract and introduction accurately claim that ViMaR introduces two-stage infer- ence framework combining temporal-difee -iffeereee value learning with margin-base based reward adjustment to improve both efficiency and output fidelity in VLM captioning. The paper explicitly claims over 4x speedup compared to VisVM, 64% human preference over Vis outputs, and consistent cross-model generalization to stronger unseen models (LLaVA-One Vision- -atiion--we22-8 and Qwen2.5-VL All claims are directly supported by experimental results in Section 4, particularly Tables 1, and Figures 1-2.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reffect how much the results can he expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
The paper discusses limitations in Section 5. The authors acknowledge that while their two-stage approach achieves superior performance. the method relies on CLIP-VIT as the process reward model. which may not perfectly capture all aspects of visual grounding. The paper also notes computational constraints were evaluated single NVIDIA RTX A6000 GPU, and inference times are mcasured under specific batch configu gurations that may vary with different hardware setups. Additionally, the method requires multiple temperature sampling (N=5, K=6) which, while efficient, still incurs overhead compared to greedy decoding.
Justification:
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model speciffcfii,,,nn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested a few datasets with few runs. In general, empirical results often depend implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures hecause fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicaall, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't It ack acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play important role in developing norms that preserve integrity the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
32

---

Question: For each theoretical result. does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [No]
The paper is empirical and architectural in nature, not theoretical. It does not present formal thcorems, lemmas, mathematical proofs. The work builds established cemporal-differene learning (Sutton, 1988) and applies to VI.M captioning without introducing new theoretical results requiring formal proofs.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross-referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof' either appear in the main paper the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of paper should be complemented by formal proof' provided in appendix or supplemental material.
Theorems and Lemmas the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: The paper provides comprchensive reproducibility information: (1) Architecture de- ViMMR is built on -Lit--NextMMsstraa-- with linear value head attached to the penultimate transformer layer; (2) Training data-792K triplets derived from 23K COCO 2017 images paired with LLaVA-150K prompts (732K train, 60K validation): (3) Hyperparameters- margin threshold =0.16 16 (justified in Appendix A.4 via percentile analysis), discount factor, learning objectives specified in Eq. 1-2; (4) Implementation details in Scction 3.1 and Appendix A.3; (5) Inference parameters- N temperatures 0.1, 0.3, 0.5, 0.7, 0.9, K=6 samples per temperature; (6) Evaluation metrics -CHAIR.
MMHal, and seven visual comprehension benchmarks clearly defined; (7) Hardware NVVIDI RTX A6000 GPU with fp16 mixed precision training. Code availability is promised for reproducibility.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible important, regardless whether the code and data are provided or not.
the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of a large language model), releasing of model checkpoint, other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submissions provide some reasonable avenue for reproducibility. which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g.. a large language model), then there should either be a way to access this model for reproducing the results a way to reproduce the model (e.g.. with open-source dataset instructions for how construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed source models, it may be that access to the model is limited in some way (e.g..
33

---

to registered users), but it should be possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code. with sufficient instructions to faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification: All datasets used are publicly available: COCO 2017 (train and validation splits), LLaVA-150K, prompts, and standard benchmarks (MM- Vet, MMBench, MMMU, Math Vista, CVBench, LLaVA- -i-lld, MMStar) documented with their sources. The authors explicitly state "Code; https:/github.com /arigihuu..cmmmnkaaai 45/ViMaR" in the paper header. Training and inference procedures are fully specified with exact hyperparameters, loss functions (Fq. 2), and architectural modifica- tions enabling reproduction. The paper provides sufficient detail (Section 3, Algorithm 1, Appendix 6.3-A.6) to implement ViMaR from scratch and reproduce reported results.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.ccm cc.pbbicc/ quides/CodeSuuben sool for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: bettps//niis.. wttpcc/cppuuu.... public// Sanidee//ooeSuumiiie www.bmmmisiiooooooioio. for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for new proposed method and baselines. If only a subset experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possihle in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
Experimental sefting/details
Question: Does the paper specify all the training and test details (e.g.. data splits, hyperparameters. how they were chosen, type of optimizer, etc.) necessary understand results?
Answer: [Yes]
Justification: The paper comprehensively specifies: (1) Data splits-732K training triplets, 60K valida- tion triplets, 1,000 images for main evaluation (Section 4.1), 500 for hallucination evaluation (Section 4.1.2). with COCO Train2017/Na2207 sources clearly identified; (2) Hyperparameters -earning rate, discount factor, margin threshold =0.16 with data-driven justification (Appendix A.4 analyzing CLIP score distribution); (3) Training procedure 4 epochs, batch size 16 per device (Appendix A.3); (4) Optimizer and ppecision-implien by J.L.VA-Nex architecture, fp16 mixed precision confirmed in Appendix A.3; (5) Temperature selection- five decoding temperatures 0, 0.3, 0.5, 0.7, 0.9 with abla- tion analysis (Appendix A.5 Table 3); (6) Reward formulation justification Appifiattio-AApendixx Appendix A.6 compares alternative reward functions with empirical validation. These details enable reproducibility and inform methodological choices.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary appreciate the results and make of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate informa- tion about the statistical significance of the experiments?
Answer: [Yes]
34

---

Justification:
The paper reports statistical significance through multiple complementary approaches:
(1) Human evaluation (Section 4. bll-dlind pairwise comparison with 300 image-prompt pairs, report- ing win rates with multiple baselines (64.0% vs VM 65.3% 5% vs CLIP-PRM, 66.0% vs BoN, 69.7%
vs greedy); (2) Automated evaluation HIPT-4o-baced pairwisc comparisons (Figure 1b) showing consistent preferences (49.3%- 33--3%3..3 73.8% across basclines); (3) Hallucination metrics (Table 1) quanti- tative comparisons with measurable reductions in R1, CHAIRs. and MMHal metries; (4) Self-training results (Table 2) perfforrmaacce improvements across multiple benchmarks with concrete percentage gains (15.87% average improvement). While error bars are not displayed in plots, the large evaluation set (1,000+ images) and multiple evaluation metrics provide robust statistical grounding.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanicd by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example. train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to library function, bootstrap, etc.)
The assumptions made should given (c.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
is OK report I-sigma error hars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CL, if the hypothesis of Normality of errors not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures
symmetric error hars that would yield results that are out of range (e.g. negative error rates).
crror bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
Experiments compute resources
Question: For each experiment, docs the paper provide sufficient information on the computer resources (type of compute workers, memory, time of exccution) needed reproduce the experiments?
Answer: [Yes]
RIX A6000 GPU with 48GB VRAM (Appendix A.3); (2) Training configuration-44 epochs with batch size 16 per device, using mixed- precision fp16 and gradient checkpointing to optimize memory; (3) Inference
Justification: The paper specifies: (1) Hardware-NNvin
in Table with average inference time per sample (108s for ViMaR vs 462s for VisVM, 668s for BoN, 62s for greedy), demonstrating computational efficiency and enabling resource planning: (4) Saanework-accelreen library with distributed training capabilities noted (Appendix A.3). These specifications enable practitioners to assess hardware requirements and deployment feasibility for their settings.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't I make it into the paper).
9. Code of ethics
Question:
Does the research conducted in the paper conform, in every respect, with the NeurIPS Code
Ethics
Answer:
[Yes]
Justification; The research conforms with the NeurIPS Code of Ethics. No personally identifiable information or private data was collected or used. All training data (COCO 2017, LLa 1A-150K) are publicly available datasets. The evaluation uses standard public benchmarks without sensitive attributes. The paper does not make claims about demographic fairness or intentionally suppress negative results.
35

---

The work focuses on improving caption quality and reducing ballucinations--nnleee inherently beneficial objectives without documented harm to individuals or groups.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure preserve anonymity (e.g., if there is special consideration due laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: paper implicitly addresses broader impacts: (1) Positive impacts reducing hallucina- tions in VLM outputs improves reliability for real-world applications (medical imaging, accessibility descriptions); enhanced visual grounding supports more accurate image understanding across domains. (2) Potential concerns-whil while not explicitly detailed, the margin-based reward adjustment and value- guided decoding represent a mechanism for shaping model behavior, which could potentially encode biases present in CLIP-ViT emheddings if training data contains skewed visual representations. (3) Self Self-training implications- uuinn model-generatee captions for further training (Section 4.2) creates feedback loops that could amplify initial biases if present. The paper's focus factual accuracy and visual grounding mitigates some risks but acknowledges no explicit safeguards against these polential negative outcomes.
Guidelines:
The answer NA means that there is no societal impact of the work performed. the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative socictal impacts include potential malicious or unintended uses (c.g., disinformation. generating fake profiles, surveillance), fairness considcrations (c.g., deploy- ment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational rescarch and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arisc when the technology is being used intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative socictal impacts, the authors could also discuss possible mitigation strategies (c.g., gated relcase of models, providing defenses in addition to attacks, mechanisms for monitor- ing misuse, mechanisms to monitor how system Iearns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible relcase of data or models that have a high risk for misuse (e.g.. pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [No]
The paper does not describe explicit safeguards for responsihle release. While all datasets used are publicly availahle and pose no privacy risks, the paper does not propose usage guidelines. restrict deployment contexts, implement safety filters for the released code value model. The work does not address potential misuse scenarios such as generating misleading captions scale or using the value model to adversarially manipulate outputs. However, the focus on hallucination reduction and visual grounding inherently promotes beneficial use cases without obvious dual-use concerns.
Justification:
Guidelines:
The answer NA means that the paper poses such risks.
36

---

Released models that have high risk for misuse or dual-use should be released with necessary safeguards allow for controlled use of the model, for example by requiring that adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets have scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
original owners assets (e.g., code, data, models), used in the paper, properly credited and are license and terms of use explicitly mentioned and properly respected?
Question: Are the
Answer: [Yes]
Justification: The paper properly credits all external assets: (1) Datasets-COOG 2017 (Lin et al.). LLaVA-150K (Liu al.), and benchmark datasets (MM-Vet, MMBench, MMMU, Math Vista, CVBench, LLaVA-Wild, MMStar) are cited with references; (2) LLaves--L
LLaVA-One ViA-OnnVision Qwen2-7B 7B, and Qwen2.5-VLL VL-3B are cited from their respective papers: (3) CLIP- cemporal-difference VIT encoder properly cited (Radford et al.): (4) Temporal difference learning framework credited to
Sutton (1988). All citations appear in the References section with full bibliographic details. The of publicly available datasets and pre-trained models complies with academic research purposes and respective licensing terms.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g.. CC-BY 4.0) should be included for each asset.
For scraped data from particular source (e.g., website), the copyright and terms of service of that source should he provided.
If assets released, the license, copyright information, and terms of use the package should be provided. For popular datasets, paperswi wwwode.hhhhee... wwt.ccom//aaac dataseta has curated licenses for some datasets. Their licensing guide can help determine the license a dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
The paper does release new pre-trained model checkpoints. ViMaR is primarily an inference-time decoding algorithm and strategy, not a novel foundation model or independent asset. The value model component is lightweight (a linear value head attached to LLa VLAVA--ext Next-Mistral-77 penultimate layer) and is trained as part of the methodology, but trained checkpoints are not provided separately. The paper promises code release via GitHub which enables reproducibility weights/checkpoints are not mentioned as being released. Therefore, while the code and repro- ducibility details are comprchensive (Sections 3.1, 3.2, Appendix A.3-A.6, Algorithm 1), no new standalone model assets requiring asset documentation are distributed.
Justification:
1asaaa/811///////// the training procedure and inference algorithm, but model
Guidelines:
answer means that the paper does not release new assets.
Researchers should communicate details of the dataset/code/mooel as part of their sub- missions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You either create anonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
37

---

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, well as details about compensation (if any)?
Answer: [No]
Justification: The paper includes human evaluation (Section 4.1, Appendix A. I) but without complete
documentation. The paper deserihes the evaluation procedure: blind pairwise comparison over 300 image-prompt pairs where annotators select preferred captions without knowing source model identity. However, the paper does include: (1) full text of instructions given to human annotators, (2) screenshots or interface screenshots showing how annotators performed task, (3) any compensation details (payment, incentives, or voluntary participation). The evaluation is in-house rather than crowd- sourced, which explains the minimal disclosure, but the checklist criteria require such documentation. A more complete submission would include annotator instructions supplementary material and clarify the evaluation setup (internal team members vs, external annotators).
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (TRB) approvals or equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [No]
Justification: The paper does not mention IRB approval or review. The human evaluation involves annotators performing caption comparison tasks, which pose minimal risk (non-invasive, no sensitive data collection, no personal information gathered). However, the paper does not explicitly state that IRB approval was obtained or that participant consent procedures were followed. Best practice would include mention of IRB exemption approval status, though the low-risk nature of the study may not require formal review depending institutional requirements. The paper could strengthen compliance by explicitly addressing this point.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applica- ble), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing. editing, formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Answer: [Yes]
Justification: The paper's 's core methodology relics heavily on LLMs as integral components: (1) Base L.MM-LLaVA-Neex-Miisrraa----
is the primary policy model for caption generation (Sec- tion 3.2, Algorithm l), providing the generation distribution from which candidates are sampled; (2) Training data-the the training triplets (yi, 11+1, I) are generated using VLaNe- with greedy decoding and temperature- seaperrauur--onttolleddd sampling to ensure diversity (Section 3.1): (3) model eval- uation ViMaR's 's value model is tested on stronger LLMs (LLaVA-Omi One--Oneiiioon Vision-Qwent Owen2-7B, Qwen2.5- VL-3B) demonstrating generalization (Table 2, Section 4.2); (4) Self- training evaluation fine-tuning experiments use LLa -A-Next-Misttall- the base model (Section 4.2). The LLM is not merely
38

---

used for writing formatting: it is central to method development, training data generation, and evaluation. This usage is clearly described throughout the methodology.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer L.M policy (https: should or should not described.
for what
39


Reviews:
[{"review_id": "Y6y8P8p5J1", "reviewer": "Reviewer_3B7b", "summary": "This paper introduces ViMaR, a novel two-stage inference-time scaling framework for VLMs. ViMaR addresses the computational expense and hallucination issues of existing VLM captioning methods by combining a temporal-difference value model with a margin-aware reward adjustment. The first stage selects a high-value coarse caption, while the second stage refines under-grounded segments. ViMaR demonstrates superior performance in generating accurate, detailed, and factual captions, achieving over 4x speedup compared to baselines. It also exhibits strong cross-model generalization and can effectively self-train VLMs.", "questions": "1. As mentioned in weakness 1, could the authors provide a more comprehensive ablation study to substantiate the effectiveness of the margin-based reward?\n\n2. As mentioned in weakness 2, could the authors provide a more detailed description of the implementation of Stage 2?\n\n3. Regarding Eq.1, the reward for \"otherwise\" ($\\tau−\\delta$) appears to be a typo. As currently written, for $\\delta=0$ and $\\delta=\\tau$, they would yield the same reward, which seems unreasonable.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "strength\n\n1. The paper is well-structured and easy to follow, presenting a clear and logical flow from problem statement to proposed solution and experimental validation.\n\n2. ViMaR demonstrates improved performance across various metrics, consistently outperforming baseline methods like VisVM in terms of caption quality, reduction in hallucinations while achieving substantial speedups.\n\n\nweakness\n\nThis work is based on VisVM, and is aimed to improve both its reward and the inference pipeline. Thw following weaknesses are related to these two aspects:\n1. While the margin-based reward is a core innovation, the paper lacks a comprehensive ablation study to fully demonstrate its effectiveness. Although a showcase highlights its advantages over the VisVM value model, more extensive quantitative comparisons are needed to rigorously validate its contribution to the overall performance gains.\n\n2. The implementation details for Stage 2 of the two-stage inference remain insufficiently described, particularly regarding how \"under-grounded or missing visual regions\" are identified (Algorithm 1, Line 8) and how supplementary segments are appended at appropriate positions (Algorithm 1, Line 15). The explanation in lines 193-194 of the main text for Algorithm 1, Line 8 is overly vague and needs further clarification.", "was_revised": true, "final_justification": "After the rebuttal, most of my concerns have been addressed. The additional ablation study validate the effectiveness of the margin-based reward, which serves as a core improvement over VisVM. I have raised my score from 3 to 4.", "rebuttal": "**Dear Reviewer 3B7b,**  \nThank you for your valuable comments and questions, which have been very helpful in clarifying and improving the presentation of our work. We address your points below:\n\n---\n\n### Question 1\n\nThe effectiveness of the margin-based reward is already partially demonstrated in our paper through multiple quantitative and qualitative analyses. Specifically, Tables 1–2 and lines (254–267 and 289–309) report significant reductions in hallucination metrics (CHAIR$_s$, MMHal) and consistent improvements in visual comprehension benchmarks when the margin mechanism is introduced. Figures 2–5 further visualize how margin-penalized segments are selectively refined, yielding better grounding compared to other methods.\n\n####  **Isolating the margin’s contribution**\n\n1. Our analysis inherently isolates the contribution of the margin-based reward: the only modification is replacing the raw similarity signal with our **margin-adjusted formulation** combined with **targeted refinement**. This change alone yields a 3–4% reduction in hallucination rates and a 15.87% average improvement across benchmarks.\n\n2. **$r_{s_i} = \\delta$**: This formulation corresponds to the baseline CLIP-PRM approach and is effectively identical to the reward design explored in the VisVM paper. We also include this variant in our own baselines (Tables 1–2), where it underperforms our margin-based reward, particularly in hallucination mitigation, confirming that a naive use of $\\delta$ is insufficient.\n\n3. To further strengthen this point, we compared our margin-based reward against a simplified variant defined as:\n\n```\nr_{s_i} =\n    δ, if δ ≥ τ\n    0, otherwise\n```\n\nThis variant removes the negative penalization component. As expected, it resulted in noticeably degraded performance and increased confusion during sentence selection. This outcome directly supports the necessity of our reward modeling design, and the quantitative evidence is provided in Table R1 (ViMaR(Reward′) entry).\n\n---\n\n#### **Extended Results**\nTo provide additional clarity, we have expanded our analysis beyond what was presented in the original submission:\n\n1. *Temperature control:* We additionally analyzed ViMaR under fixed temperature decoding (e.g., T = 0.2) versus BoN and our multi-temperature setup. Results in Table R1 confirm that diverse temperature sampling is a key factor in ViMaR’s superior performance relative to Best-of-N baselines.\n\n2. *Cross-model evaluation:* We tested ViMaR on the open-source **Qwen2.5-VL-3B** model in Table R1, which already exhibits strong baseline grounding. Despite this, ViMaR still provided measurable improvements in hallucination reduction and descriptive richness, confirming the generality of our approach.\n\n---\n\nAbove analyses, combined with the results already in the paper, rigorously validate both our architecture and the margin-based reward formulation. They demonstrate that the margin not only improves hallucination mitigation but also enhances cross-model generalization and robustness across decoding strategies.\n\n---\n\n### Table R1\n\nViMaR-guided decoding consistently enhances visual comprehension and mitigates hallucinations across multiple benchmarks, outperforming alternative strategies.\n\n| Base Model | Variant               | MM-Vet ↑ | MMBench ↑ | MMMU ↑ | MathVista ↑ | CVBench ↑ | LLAVA-W ↑ | MMStar ↑ | CHAIRs ↓ | CHAIRi ↓ | MMHal ↑ | MMHal rate ↓ |\n|------------|----------------------|----------|-----------|--------|-------------|-----------|-----------|----------|----------|----------|---------|--------------|\n| **Qwen2.5-VL-3B** | Original          | 61.8 | 79.1 | 31.5 | 62.3 | 72.1 | 88.4 | 55.9 | 18.5 | 3.7 | 3.82 | 0.32 |\n|            | ViMaR               | 62.3 | 81.2 | 33.4 | 64.5 | 72.8 | 89.2 | 56.3 | 17.2 | 3.2 | 3.94 | 0.28 |\n| **LLaVA-Next-Mistral-7B** | ViMaR(Reward′)   | 48.8 | 77.6 | 36.7 | 41.4 | 70.2 | 79.2 | 38.7 | 21.5 | 4.1 | 3.44 | 0.41 |\n|            | ViMaR (Temp=0.2)    | 49.7 | 77.3 | 37.1 | 42.5 | 70.4 | 79.5 | 38.9 | 21.1 | 3.97 | 3.68 | 0.39 |\n|            | ViMaR               | **49.8** | **78.2** | **37.4** | **42.5** | **70.7** | **79.9** | **39.3** | **20.8** | **3.9** | **3.73** | **0.38** |\n\n---\n\n### Question 2\n\nWe note the reviewer’s request for additional clarity regarding Stage 2 of our two-stage inference framework. While Stage 1 identifies the highest-value caption via best-of-N generation guided by the value model $V_ρ$, Stage 2 is specifically designed to *refine only visually under-grounded segments* rather than regenerate the entire caption, reducing compute overhead while improving fidelity.\n\n**1. Segment Identification.** After Stage 1 generates the coarse caption $y^*$, we compute sentence-level value scores for each segment $y_i$ relative to the image. Segments falling below the calibrated margin threshold 1.87 (similarly calculated as τ) are flagged as “under-grounded” and removed from the coarse caption. Additionally, missed visual entities (e.g., objects detected in image features but absent from the caption) are flagged via cross-attention coverage analysis.\n\n**2. Candidate Generation.** For each flagged segment, we condition decoding on the preceding accepted segments $y_{<i}$ and image $I$, sampling K = 6 candidates per temperature from the fixed set {0.1, 0.3, 0.5, 0.7, 0.9} (same as Stage 1). These candidates are short continuations targeted to replace only the flagged segment.\n\n**3. Scoring and Selection.** Each candidate is scored using the value model $V_ρ(s,I)$, which is trained using temporal-difference value estimation (Sec. 3.1). The highest-value candidate replaces the flagged segment in the caption.\n\n**4. Iterative Refinement and Termination.** This process iterates until all flagged segments exceed the margin threshold or no further under-grounded regions are detected. Because refinement operates at the segment level rather than paragraph level, inference cost is reduced by ~4× compared to VisVM (Table 1).\n\nAdditional pseudocode for this procedure is provided in Algorithm 1 (lines 7–15), and the same beam width and temperature settings are used across both stages for fair comparison to baselines.\n\n---\n\n### Question 3\nWe thank the reviewer for pointing out this typo. Indeed, there was a typographical error in the “otherwise” branch of Eq. 1 in the draft. The intended reward formulation penalizes under-grounded predictions (low δ) rather than assigning them the same reward as well-grounded ones. Specifically, when δ < τ, the reward should be computed as:  \n\n`r_{s_i} = δ − τ`  \n\nwhich yields a negative value and therefore acts as a penalty within the temporal-difference objective. This ensures that low-confidence (under-grounded) sentences are explicitly down-weighted during value estimation, encouraging the model to learn a sharper separation between well-grounded and poorly grounded captions.\n\nFor clarity, the corrected Eq. 1 is:\n```\nr_{s_i} =\nδ, if δ ≥ τ\nδ − τ, if δ < τ\n```\n\nHere, δ ∈ [0, 1] denotes the CLIP similarity score, and τ = 0.16 is a calibrated margin derived from the empirical score distribution (see Appendix D). We will update the manuscript to reflect this corrected equation and prevent ambiguity."}, {"review_id": "5uSgXYRN3k", "reviewer": "Reviewer_wMfE", "summary": "This paper proposes a two-stage test-time scaling method. First, it trains a reward model using CLIP similarity. Then, using this reward, it first scores coarse-grained captions and subsequently scores fine-grained captions.", "questions": "1. This work claims that its method is effective for VLMs. However, the paper only validates the method on LLaVA, so it is unknown whether it works on other models. I believe evaluating more VLM models would make the work more solid, such as open-source models like Qwen. Besides, this method could even be applied to closed-source models like GPT-4o. If possible, it would be worth trying.\n\n2. The CLIP image-text similarity does not accurately reflect the quality of captions, and I believe there are two reasons for this. First, the CLIP similarity itself is not precise. When calculating CLIP similarity between an image and different texts, the matching text may not necessarily receive the highest score. Even if it does, its score is often not significantly higher than that of non-matching options. Second, CLIP similarity is usually used to measure the match between an image and a full caption, but in this paper, it is used to evaluate the similarity between a single sentence (which only describes a small detail of the image) within a longer caption and the image. I think there is a huge gap here. In summary, I believe using CLIP similarity for this task has significant issues. I suggest replacing it with other, stronger similarity measurement methods.\n\n3. There is no relevant discussion on the selection of the threshold in Equation 1.\n\n4. The writing of this paper is not clear enough. I believe adding some illustrative figures could help readers better understand the method.", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strength**\n\nThis work reduces computational cost by using a two-stage approach.\n\n**Weaknesses**\n\nThis work has a fundamental flaw in that it uses CLIP similarity as the loss to train the reward model. Although CLIP is a groundbreaking work, its image-text similarity is not very accurate. Especially since this paper deals with paragraph-length captions, which are quite long, the CLIP similarity between a single sentence and the image does not provide meaningful information.", "was_revised": true, "final_justification": "I apologize for the late response. I have carefully read the author's rebuttal and sincerely appreciate their detailed reply. I believe that the vast majority of the issues have been addressed. Although I still think that using CLIP is not entirely appropriate, this does not diminish the contribution of the paper. I hope the authors will have the opportunity to replace it with a better text encoder in the future. I will raise my score to 4.", "rebuttal": "**Dear Reviewer wMfE,**  \nThank you for your valuable comments and questions, which have been very helpful in clarifying and improving the presentation of our work. We address your points below:\n\n---\n\n### Question 1\n\nOur work already demonstrates cross-model generalization beyond LLaVA. In addition to LLaVA-Next-Mistral-7B, we explicitly evaluate ViMaR on *LLaVA-OneVision-Qwen2-7B* (Table 2, lines 305–309), showing consistent improvements across all benchmarks, including hallucination metrics (CHAIR, MMHal).\n\nTo further strengthen this point, we have extended our evaluation to an additional open-source VLM, *Qwen2.5-VL-3B*. Results (provided in Table R1) similarly confirm that ViMaR delivers substantial gains in both fidelity and hallucination mitigation, underscoring its model-agnostic applicability.\n\nFinally, we emphasize that ViMaR operates purely at inference time and does not require access to model weights (as also noted in lines 52–56, where we highlight its plug-and-play generalizability across VLM architectures). This property makes the method directly applicable even to closed-source APIs such as GPT‑4o. While such evaluations are technically feasible, they are prohibitively costly to perform at scale; therefore, we prioritized comprehensive open-source benchmarks for reproducibility and resource efficiency. We will clarify this practical consideration explicitly in the final version.\n\n---\n\n### Table R1\n\nComparison between the original Qwen baseline and ViMaR-guided decoding.\n\n| Base Model | Variant | MM-Vet ↑ | MMBench ↑ | MMMU ↑ | MathVista ↑ | CVBench ↑ | LLAVA-W ↑ | MMStar ↑ | CHAIRs ↓ | CHAIRi ↓ | MMHal ↑ | MMHal rate ↓ |\n|------------|---------|----------|-----------|--------|-------------|-----------|-----------|----------|----------|----------|---------|--------------|\n| **Qwen2.5-VL-3B** | Original | 61.8 | 79.1 | 31.5 | 62.3 | 72.1 | 88.4 | 55.9 | 18.5 | 3.7 | 3.82 | 0.32 |\n|                | ViMaR    | 62.3 | 81.2 | 33.4 | 64.5 | 72.8 | 89.2 | 56.3 | 17.2 | 3.2 | 3.94 | 0.28 |\n\n---\n\n### Question 2\n\nWe thank the reviewer for raising this important point. We acknowledge that naive use of CLIP similarity can be limited in both precision and its ability to capture fine-grained, sentence-level details (as reflected in Tables~1,2 CLIP-PRM study). Our approach specifically addresses these concerns in three ways:\n\n**1. Temporal-Difference Value Model Beyond Raw CLIP Scores:**  \nWe do not rely on raw CLIP similarity alone. Instead, we use CLIP scores to calculate a *Margin-Based Reward Adjustment* within a temporal-difference (TD) value model (Sec. 3.1) that predicts long-term reward across the entire caption trajectory. This approach smooths local noise in individual scores by considering both the immediate alignment of a sentence and its downstream influence on subsequent sentences. Empirically, this yields more discriminative value estimates (Fig. 2), avoiding the instability noted with raw CLIP usage.\n\n**2. Margin-Based Reward Adjustment for Fine-Grained Segments:**  \nTo penalize under-grounded predictions (low δ), we introduce a margin-based penalty (Eq. 1) that calibrates the reward for partial sentences. Rather than treating all low scores equally, sentences below the calculated threshold τ (see Appendix D) are penalized proportionally according to their similarity scores. This mechanism effectively normalizes the CLIP reward score, allowing it to distinguish visually grounded details without over-penalizing neutral content.\n\n**3. Empirical Justification and Ablation Results:**  \nWe conducted extensive evaluations (Tables 1–2) demonstrating that our CLIP-based value model, when combined with margin calibration and two-stage refinement, consistently reduces hallucination rates (CHAIR_s from 26.2% to 23.1%) and improves factual richness compared to VisVM and other baselines. Importantly, this improvement holds even when applied cross-model (e.g., LLaVA-OneVision-Qwen2-7B or Qwen2.5-VL-3B) in the main manuscript Table 2 and Table R1, suggesting robustness beyond any single similarity metric.\n\nIn summary, while raw CLIP similarity alone is imperfect, our value-guided framework addresses its shortcomings via TD learning, margin calibration, and segment-level refinement, leading to demonstrably better grounding and efficiency over prior methods.\n\n---\n\n### Question 3\n\nThis point is already addressed in our paper (Appendix D, lines 547–563). Specifically, we describe that the threshold τ was derived from the empirical CLIP similarity distribution on the training set (17th percentile), ensuring that only visually under-grounded sentences receive a penalty.\n\nAs further detailed in lines 147–151 (Equation 1), τ serves as the calibrated margin that determines when and how strongly low-confidence sentences are penalized: sentences above τ are rewarded proportionally to their CLIP alignment, while those below τ incur a penalty proportional to the gap. This mechanism is central to reducing hallucination without over-penalizing neutral or background details, and its role is explicitly described in the cited section. We further note that small variations in τ yield minimal performance change, indicating robustness.\n\n---\n\n### Question 4\n\nWe already included some visual aids to clarify the method and results. These figures were designed to complement the algorithmic description in Section 3 (Algorithm 1) and provide both conceptual and qualitative clarity. We agree that adding a high-level schematic of the two-stage pipeline (coarse caption generation and targeted refinement) could further improve readability, and we plan to include such a simplified diagram in the camera-ready version for additional clarity."}, {"review_id": "cnpe4UXA3p", "reviewer": "Reviewer_vUxp", "summary": "This paper proposes a method for training a value function using triplet data consisting of an image, a current sequence, and a subsequent sequence. During inference, a beam search-like decoding strategy is employed to generate captions, which the authors claim helps mitigate hallucination.", "questions": "see Strengths And Weaknesses.", "limitations": "N/A", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. The proposed method appears interesting.\n2. The experimental results seem promising.\n\nWeaknesses:\nThere are some unnatural (or unreasonable) processes of the  proposed method need to be clarified:\n1. Why is the reward (in line 151) defined in the state? To the best of my knowledge, in the traditional MDP of RL, a valid reward is defined with a pair of state and action. Actually, \"the reward\" of a state without action is called a value function in the traditional RL.\n2. The rationale behind the definition of $r_{s_i}$ in Eq. (1) remains unclear. This design introduces a discontinuity in $r_{s_i}$ when $\\delta$ approaches $\\tau$. More critically, it causes an abrupt change in the gradient $\\frac{\\partial r_{s_i}}{\\partial \\delta}$ from $-1$ to $1$ near $\\delta = \\tau$, which could potentially lead to significant instability during training. To address this concern, an ablation study is necessary to justify this design choice. In particular, it would be helpful to compare Eq. (1) with one or more alternative formulations such as:\n\n* (a) $r\\_{s\\_i} = \\delta$\n* (b) $r_{s_i} = \\max(\\delta, \\tau)$\n* (c) $r_{s_i} = \\delta$ if $ \\delta \\geq \\tau $,  and $r_{s_i} = 0$ if $ \\delta < \\tau  $\n\nSuch comparisons would provide empirical evidence for the effectiveness and stability of the proposed formulation.\n\n4. The definition of the value function $V$ in Eq. (2) appears unnatural. Intuitively, the next state following $s_i$ should at least incorporate the information from $y_i$, $y_{i+1}$, and $I$. However, the formulation $V_{\\rho}(y_{i+1}, I)$ in Eq. (2) (potentially) suggests that the value of the next state depends only on $y_{i+1}$ and $I$, while ignoring $y_i$. This raises concerns about whether the state transition is properly captured, and whether important contextual information from $y_i$ is being omitted.\n\n5. Using different temperatures in BoN as well as in the proposed method is interesting. However, there are no comparisons with the fixed temperatures (say 0.2 or 0.6) in both BoN and the proposed method to verify whether this design of different temperatures really works.", "was_revised": true, "final_justification": "(1) The additional table R2 in the rebuttal clearly states the superiority of the proposed reward design over other alternative reward designs. Based on this point, I am increasing the score of significance from \"fair\" to \"good\".\n\n(2) However, both table R1 and table R2 have suggested that the proposed diverse temperature design has no obvious performance gap compared to the traditional fixed temperature design, and this gap is within the range of random error. Note that diverse temperatures would even introduce more hyperparameters, which make it harder to apply in real applications.  Hence, the concern about the proposed diverse temperature still exists. Therefore, I tend to maintain the score of 3.\n\nThat said, I would not get frustrated if this paper is accepted.", "rebuttal": "**Dear Reviewer vUxp,**  \nMany thanks to your professional, detailed, and valuable reviews. We address your concerns one by one below.\n\n---\n\n### Question 1\n\nWe clarify that the reviewer’s concern arises from a misinterpretation of our formulation. In our framework, the reward in Eq. (1) is not purely “state-based” as implied. It is computed for the current state–action pair $(s_i, y_i)$, where $s_i = (y_i, I)$, with $y_i$ being the current sentence (including previously generated context) and $I$ being the input image. This reward $r_{s_i}$ is precomputed using the CLIP-based margin adjustment and does not involve any learning.\n\nThe training objective in Eq. (2) then uses this fixed reward to optimize the value function $V_ρ$ via temporal-difference learning, allowing $V_ρ$ to predict cumulative future rewards. This formulation is consistent with the standard MDP paradigm; our notation simply omits explicit action dependency for brevity.\n\n---\n\n### Question 2\n\nIn our framework, the reward $r_{s_i}$ defined in Eq. (1) is *precomputed* using the CLIP similarity with margin adjustment and is not a learnable function through which gradients are propagated. The actual training objective is Eq. (2), where $r_{s_i}$ is treated as a fixed target and combined with the predicted value $V_ρ$ in a temporal-difference (TD) loss. Therefore, the reviewer’s concern regarding a discontinuous gradient $\\frac{\\partial r_{s_i}}{\\partial \\delta}$ at $\\delta=\\tau$ does not apply, as no gradient flows through Eq. (1) during optimization.\n\nRegarding the design choice: the sharp margin is intentional. Our goal is to strongly penalize hallucinated captions (low $\\delta$) and reward grounded ones (high $\\delta$), making the separation between positive and negative cases explicit. This simple thresholding provides a clear and interpretable signal for the value model, and our ablations (Appendix D, lines 547–563) confirm robustness: small variations in $\\tau$ do not destabilize training or degrade performance.\n\nIn summary, the margin-based reward is a fixed supervisory signal rather than a trainable component, and the reported results (Tables 1–2) empirically validate its stability and effectiveness for hallucination mitigation.\n\n---\n\n### Question 3\n\nWe appreciate the reviewer’s suggestion to compare Eq. (1) with alternative formulations. We clarify our rationale and provide evidence as follows:\n\n**(a) $r_{s_i} = \\delta$:**  \nThis formulation corresponds to the baseline CLIP-PRM approach and is effectively identical to the reward design explored in the VisVM paper. We also include this variant in our own baselines (Tables 1–2), where it underperforms our margin-based reward, particularly in hallucination mitigation, confirming that a naive use of $\\delta$ is insufficient.\n\n**(b) $r_{s_i} = \\max(\\delta, \\tau)$:**  \nThis design removes penalization for scores below $\\tau$ by mapping all low-confidence sentences to the same value $\\tau$, effectively treating mildly and strongly hallucinated sentences identically. This contradicts our objective of sharply distinguishing grounded and hallucinated captions. Empirically, this leads to weak gradients for low-quality candidates and fails to suppress hallucinations. Preliminary experiments confirmed inferior performance compared to our chosen design.\n\n**(c) $r_{s_i} = \\delta$ if $\\delta \\ge \\tau$, else $0$:**  \nWhile this variant is superficially similar to our margin formulation, it removes negative penalization entirely and, critically, assigns the same reward to both mildly and strongly hallucinated sentences (all mapped to zero). This lack of differentiation weakens the model’s ability to prioritize corrections and leads to poorer sentence selection. Our experiments confirm this: performance drops noticeably and hallucination rates increase (results provided in Table R1), directly validating the necessity of our margin-based reward design.\n\nThus, our design explicitly penalizes under-grounded sentences rather than merely ignoring them, yielding significantly better alignment and stability, as supported by the quantitative results in Tables 1–2 and Table R1.\n\n---\n\n### Question 4\n\nThe reviewer’s concern arises from a misinterpretation of our notation rather than the actual implementation. In our framework, the value function $V_ρ$ is computed over the *entire state*, which by definition (lines 133–136) includes the image $I$ *and all previously generated sentences* $y_{<i}$ (including $y_i$). Thus, when we write $V_ρ(y_{i+1}, I)$ in Eq. (2), this is a shorthand representation: $y_{i+1}$ is conditioned on the full prefix $y_{\\leq i}$ and $I$, and the value function estimates the expected cumulative reward from this extended state.\n\nIn other words, our method does *not* discard contextual information from $y_i$; rather, it explicitly incorporates the full caption history up to step $i+1$ along with the image features. This design ensures proper state transitions and preserves long-range dependencies, which is essential for accurate paragraph-level captioning. We will clarify this explicitly in the final version.\n\n---\n\n### Question 5\n\nOur paper already describes the temperature strategy (Sec. 4.1): both BoN and ViMaR generate candidates using a fixed set of diverse temperatures {0.1, 0.3, 0.5, 0.7, 0.9} to promote diversity and capture both fine-grained and global details. This multi-temperature design is essential to our two-stage framework — Stage 1 benefits from broad candidate diversity, and Stage 2 selectively refines weak segments by leveraging candidates with varied entropy levels.\n\nTo directly address the reviewer’s suggestion, we performed additional experiments using *fixed* temperatures (e.g., T = 0.2) for both BoN and ViMaR. The results in Table R1 show consistent performance degradation: hallucination rates increase and caption richness declines compared to our multi-temperature setup. This is expected — fixed low temperatures (e.g., 0.2) lead to overly conservative, low-diversity captions missing fine details, while fixed higher temperatures (e.g., 0.6) increase diversity but may introduce hallucinations. Our multi-temperature scheme balances these extremes by mixing low-entropy and high-entropy samples, enabling the value model to select globally grounded yet detailed captions.\n\nThus, the proposed multi-temperature approach is not arbitrary: it is explicitly designed to enhance candidate diversity without sacrificing grounding, and both our main results (Tables 1–2) and new experiments confirm its superiority.\n\n---\n\n### Table R1\n\nViMaR-guided decoding consistently enhances visual comprehension and mitigates hallucinations across multiple benchmarks, outperforming alternative strategies.\n\n| Base Model | Variant            | MM-Vet ↑ | MMBench ↑ | MMMU ↑ | MathVista ↑ | CVBench ↑ | LLAVA-W ↑ | MMStar ↑ | CHAIRs ↓ | CHAIRi ↓ | MMHal ↑ | MMHal rate ↓ |\n|------------|--------------------|----------|-----------|--------|-------------|-----------|-----------|----------|----------|----------|---------|--------------|\n| **Qwen2.5-VL-3B** | Original       | 61.8 | 79.1 | 31.5 | 62.3 | 72.1 | 88.4 | 55.9 | 18.5 | 3.7 | 3.82 | 0.32 |\n|              | ViMaR            | 62.3 | 81.2 | 33.4 | 64.5 | 72.8 | 89.2 | 56.3 | 17.2 | 3.2 | 3.94 | 0.28 |\n| **LLaVA-Next-Mistral-7B** | ViMaR(Reward′) | 48.8 | 77.6 | 36.7 | 41.4 | 70.2 | 79.2 | 38.7 | 21.5 | 4.1 | 3.44 | 0.41 |\n|              | ViMaR            | **49.8** | **78.2** | **37.4** | **42.5** | **70.7** | **79.9** | **39.3** | **20.8** | **3.9** | **3.73** | **0.38** |\n| **LLaVA-Next-Mistral-7B** | BoN(Temp=0.2)  | 45.6 | 75.4 | 34.5 | 39.9 | 66.1 | 77.0 | 36.3 | 31.2 | 5.6 | 2.98 | 0.51 |\n|              | ViMaR(Temp=0.2)  | 49.7 | 77.3 | 37.1 | 42.5 | 70.4 | 79.5 | 38.9 | 21.1 | 3.97 | 3.68 | 0.39 |"}, {"review_id": "joaloEIouy", "reviewer": "Reviewer_ATYx", "summary": "The paper proposes a new two-stage value-guided captioning strategy (ViMaR) that improves the accuracy and detail of vision-language model outputs while reducing hallucinations. The authors also show that the captions generated using ViMaR are also used to self-train the base model, leading to consistent improvements across multiple benchmarks.", "questions": "Please address weaknesses. I believe the manuscipt's technical contribtuions are valuable, and it's results impressive, but would like to see some more analysis in both ablations and failure cases, in order to be more useful to the community at large. Would be open to increasing my score accordingly.", "limitations": "Failure cases of the method are not properly discussed", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "****Strengths****\n\n1. The paper introduces a two-stage decoding approach that improves captions by selectively refining weak segments instead of re-generation, making the process both efficient and accurate.\n\n2. The margin-based reward seems to be a simple and effective method for training ViMar as an effective value model to penalise weak visual captions\n\n3. The evaluation is well-rounded, using human studies and GPT-4o comparisons to clearly support the method’s effectiveness.\n\n****Weaknesses****\n\n1. Ablations studies seem to be lacking for example on various hyperparameter choices  like the number of candidates in each stage, the temperature settings, or the margin threshold τ used during value model training, making it hard to assess how robust the method is to these design choices\n\n2. There is lack of discussion on failure cases, which is important to understand for these methods.\n\n3. It is not very clear how weak sections of the caption generated in the first step of the inference search are identified. The paper mentions low confidence predictions and missing objects, but does not elaborate further. Given this is a key step in inference search, a more in depth discussion with a few examples (even in the supplementary) would be great to see to better understand.", "was_revised": true, "final_justification": "I appreciate the information provided on the hyperparameters and the new results given. I hope that the hyperparameter sweep will be added in future versions. I also appreciate the clarifications the authors have provided regarding the identification of weak segments in the captions generated in Stage 1 of the algorithmn, and I would request them to add this clarification in future versions as well.\n\nWith regards to the failure cases however, I still do not believe that they have been discussed adequately. The lines mentioned (329-354) do not delve into failure cases, but rather provide a discussion on a case study highlighting the core differences between ViMAR and VisVM, and figures 2-5 mentioned seemed to be cherry picked examples where ViMAR has no hallucinations, while the other methods shown hallucinate (marked in red).\n\nThe authors have since provided an adequte discussion on the failure cases, and as result, I have ****increased my score to 5: Accept**** as I believe this manuscript does have technical merit, and the results are quite promising. I appreciate the authors effort in the rebuttal and I thank them for providing the new results and discussions.", "rebuttal": "**Dear Reviewer ATYx,**  \nMany thanks for your detailed and valuable reviews. We address your concerns one by one below.\n\n---\n\n### Question 1\n\nWe appreciate the reviewer’s interest in more extensive hyperparameter ablations. Our design choices were carefully guided by a trade-off between caption quality and inference efficiency, and several related analyses are already provided in the main paper and appendix. To further address this concern, we have conducted additional experiments, summarized below:\n\n**1. Number of candidates K:**  \nAs stated in Sec. 4.1, we fixed K = 6 candidates per temperature to achieve a practical trade-off between performance and compute cost. Increasing K beyond 6 does improve quality slightly but scales inference cost linearly, making it impractical for deployment (e.g., K = 12 nearly doubles runtime with marginal gains). We provide additional results with K = 2 in Table R1, which show a clear performance drop.\n\n**2. Temperature settings:**  \nOur multi-temperature strategy {0.1, 0.3, 0.5, 0.7, 0.9} (Sec. 4.1) was chosen to ensure diversity while maintaining grounding. In Table R1, new fixed-temperature experiments (e.g., T = 0.2) show degraded caption quality and higher hallucination rates, validating the importance of varied entropy in candidate generation.\n\n**3. Margin threshold τ:**  \nThe rationale and calibration for τ are explicitly detailed in Appendix D. We select τ at the 17–20 percentile of the CLIP similarity distribution to penalize weakly grounded sentences without over-penalizing borderline cases. This percentile-based choice ensures robustness across datasets, as shown by our stability analysis (Appendix D, lines 547–563).\n\nDue to time constraints, we prioritized core results, but we agree more ablations (e.g., full K and τ sweeps) would be valuable and plan to include them in the camera-ready version. Preliminary tests already indicate that our method is stable across reasonable ranges of these hyperparameters.\n\n---\n\n### Question 2\n\nThe failure modes and their mitigation are already discussed in multiple parts of our paper. Specifically, Section 5 (Observations and Limitations, lines 329–354) and Figures 2–5 provide qualitative analyses comparing ViMaR and VisVM, explicitly marking grounded (green) versus hallucinated (red) segments. These examples illustrate the primary failure cases of prior methods—such as misattributing visual attributes or overlooking peripheral details—and demonstrate how our two-stage refinement and margin-bas...\n\nImportantly, our margin-aware reward directly addresses these failure cases by sharply penalizing low-confidence hallucinated segments and refining them selectively in Stage 2, avoiding costly full caption regeneration. While residual errors (e.g., in highly ambiguous scenes) are acknowledged, they occur far less frequently and are highlighted in our qualitative case studies (Appendix E).  \n\nCould the reviewer please clarify which specific failure cases they believe are missing? We would be happy to more detailed discussion of these cases.\n\n---\n\n### Question 3\n\nWe thank the reviewer for pointing out the need for additional clarity regarding inference search of our two-stage inference framework. While Stage 1 identifies the highest-value caption via best-of-N generation guided by the value model $V_ρ$, Stage 2 is specifically designed to *refine only visually under-grounded segments* rather than regenerate the entire caption, which reduces compute overhead while improving fidelity.\n\n**1. Segment Identification:** After Stage 1 generates the coarse caption y*, we compute sentence-level value scores for each segment y_i relative to the image. Segments falling below the calibrated margin threshold 1.87 (similarly calculated as τ) are flagged as “under-grounded” and removed from the sentence. Additionally, missed visual entities (e.g., objects detected in image features but absent from the caption) are flagged via cross-attention coverage analysis.\n\n**2. Candidate Generation:** For each flagged segment, we condition decoding on the preceding accepted segments y_{<i} and image I, sampling K = 6 candidates per temperature from the fixed set {0.1, 0.3, 0.5, 0.7, 0.9} (same as Stage 1). These candidates are short continuations targeted to replace only the flagged segment.\n\n**3. Scoring and Selection:** Each candidate is scored using the value model $V_ρ(s,I)$, which is trained using temporal-difference value estimation (Sec. 3.1). The highest-value candidate replaces the flagged segment in the caption.\n\n**4. Iterative Refinement and Termination:** This process iterates until all flagged segments exceed the margin threshold or no further under-grounded regions are detected. Because refinement operates at the segment level rather than paragraph level, inference cost is reduced by ~4× compared to VisVM (main manuscript Table 1).\n\nAdditional pseudocode for this procedure is provided in Algorithm 1 (lines 7–15), and the same beam width and temperature settings are used across both stages for fair comparison to baselines.\n\n---\n\n### Table R1\n\nAblation results comparing ViMaR with default settings versus reduced diversity settings (T = 0.2, K = 2).\n\n| Base Model | Variant        | MM-Vet ↑ | MMBench ↑ | MMMU ↑ | MathVista ↑ | CVBench ↑ | LLAVA-W ↑ | MMStar ↑ | CHAIRs ↓ | CHAIRi ↓ | MMHal ↑ | MMHal rate ↓ |\n|------------|----------------|----------|-----------|--------|-------------|-----------|-----------|----------|----------|----------|---------|--------------|\n| **LLaVA-Next-Mistral-7B** | ViMaR (Temp=0.2) | 49.7 | 77.3 | 37.1 | 42.5 | 70.4 | 79.5 | 38.9 | 21.1 | 3.97 | 3.68 | 0.39 |\n|                          | ViMaR (K=2)     | 48.5 | 77.1 | 36.8 | 41.7 | 70.2 | 78.8 | 38.6 | 21.5 | 4.1  | 3.65 | 0.40 |\n|                          | **ViMaR**       | **49.8** | **78.2** | **37.4** | **42.5** | **70.7** | **79.9** | **39.3** | **20.8** | **3.9** | **3.73** | **0.38** |"}]

Author rebuttal:

Meta-review: This paper introduces ViMaR, a two-stage, value-guided inference framework for vision–language model (VLM) captioning. The first stage selects a coarse caption using a temporal-difference-trained value model, while the second stage selectively refines weak segments. 

Most reviewers initially raised concerns about ablations, notation clarity, and the reliance on CLIP similarity. The authors addressed these issues through detailed rebuttals, additional experiments, and clarifications, which led three reviewers to increase their scores. Reviewer vUxp maintained a borderline rating (3) due to reservations about the temperature design but explicitly acknowledged the value of the paper, particularly its reward design, and stated they would not oppose acceptance. 

Overall, the reviewer consensus is positive and recognizes the paper’s technical merit and practical contributions. In particular, the work tackles the important challenges of hallucination and efficiency in VLM captioning, and the proposed two-stage targeted refinement is simple, effective, and efficient. Given the overall positive ratings and consensus, the AC sees no reason to overturn the reviewers’ recommendations and therefore supports acceptance of this paper.
Decision: Accept (poster)