More Thinking, Less Seeing? Assessing Amplified Hallucination in Multimodal Reasoning Models
Authors: Zhongxing Xu
Chengzhi Liu
Qingyue Wei
Juncheng Wu
James Zou
Xin Eric Wang
Yuyin Zhou
Sheng Liu
Venue: NeurIPS 2025 poster ()
Keywords: Reasoning model
Multimodality
Hallucination
Primary area: deep_learning

Abstract: Test-time compute has empowered multimodal large language models to generate extended reasoning chains, yielding strong performance on tasks such as multimodal math reasoning. However, we observe that this improved reasoning ability often comes with increased hallucination: as generations become longer, models tend to drift away from image-grounded content and rely more on language priors. Attention analysis reveals that longer reasoning chains reduce focus on visual inputs, contributing to hallucination. To systematically study this phenomenon, we introduce RH-AUC, a metric that quantifies how a model's perception accuracy changes with reasoning length, enabling evaluation of whether the model preserves visual grounding while reasoning. We also release RH-Bench, a diagnostic benchmark covering diverse multimodal tasks, designed to jointly assess the balance of reasoning ability and hallucination. We find that (i) larger models generally exhibit a better balance between reasoning and perception; (ii) reasoning and perception balance depends more on the types and domains of the training data than its volume. Our findings highlight the need for evaluation frameworks that account for both reasoning quality and perceptual reliability.

More Thinking, Less Seeing? Assessing Amplified Hallucination in Multimodal Reasoning Models
Zhongxing Xu
Chengzhi Liu
Qingyue Wei
Juncheng Wu
James Zou'
Xin Eric Wang?
Yuyin Zhou
Sheng Liu'
Stanford University PUC Santa Barbara
UC Santa Cruz
chengzhi Bucsb. edu zhongxingxu100maill. com
@htenl@stanfordcco
Abstract
Test-time compute has empowered multimodal large language models to generate extended reasoning chains, yielding strong performance tasks such as multi- modal math reasoning. However, we observe that this improved reasoning ability often comes with increased hallucination: generations become longer, models tend to drift away from -gguunde content and rely more on language priors. Attention analysis reveals that longer reasoning chains reduce focus visual inputs, contributing to hallucination. To systematically study this phenomenon, we introduce RH-AUC, metric that quantifies how model's perception accuracy changes with reasoning length, enabling evaluation of whether the model preserves visual grounding while reasoning. We also release RH-Bench, a diagnostic bench- mark covering diverse multimodal tasks, designed to jointly assess the balance of reasoning ability and hallucination. We find that (i) larger models generally exhibit a better balance between reasoning and perception; (ii) reasoning and perception balance depends more on the types and domains of the training data than its volume. Our findings highlight the need for evaluation frameworks that account for both reasoning quality and perceptual reliability.
Introduction
Large reasoning models scale test-time computation to improve complex reasoning. These models [ 7 [22 2 generate longer outputs and engage in deeper reasoning before producing final answers, resulting in more comprehensive solutions for complex mathematical and scientific problems. This paradigm has been extended to multimodal large language models: non-reasoning base models are supervised finetuned (SFT), or finetuned with reinforcement learning (RI.) to obtain strong reasoning ability [36, 56 10 49 54 21 demonstrating exceptional capabilities in multimodal reasoning tasks, particularly in domains like mathematical problem solving.
Most existing studies on multimodal reasoning models focus on enhancing reasoning performance, with limited attention paid to perception-focusee tasks. As illustrated in Figure [T although the reasoning model generates extended reasoning chain in visual question answering, its answer is largely driven by language priors rather than visual evidence, leading to hallucination. Our empirical study reveals consistent and significant finding: although reasoning models can generate more detailed reasoning chains, they introduce more hallucinations perception-focusee tasks than the non-reasoning counterparts, shown in Figure Pp
Through attention analysis, investigate the decrease of attention on visual tokens in multimodal reasoning models, which exacerbates visual hallucinations. The reasoning model allocates signif- icantly less attention to visual tokens compared its non-reasoning counterpart, while directing more attention to the instruction tokens. This bias increases reliance on language priors and amplifies
https://mlrm httpuuuummmmaaau....uaaa github. 10/ Work partially done while ZX was visiting Stanford.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Perception Task
No, image vtatin sports in
Qwen2 (Base)
Perception (%6
$5.0
Qvew2.S-VL-3B
(Base)
64.0
(wee2.5.PP
(Bave)
Thind Li VL-7B
with better, cotcher,
ball image?
sports
the ssscball roof is soible
02.0
MMM-Fundo-771
R1- Vitoon (Reasening)
$4.0
Ocan-ki-331
ovayyarg
60.0
image itself Although spectfic
mestion of spente ball made the
$2.
Can-RFFT-3
SpasVTTiikke-78
contect stranply mmples inssive
beseball
ball vitiise siewerr Ves there is
sperts bell in the image s/Ansver>
45.0
LLM-RII-3
40.1
42.8
46:11
4411 46 40d
s0n 52. 5401 MED 380
Reasoning Accurney (%)
Reasoning Accuracy (%6
(b)
Figure I: (a) Example of outputs from reasoning model and a non-reasoning model perception task. Red highlights indicate visual hallucination. Multimodal reasoning models are generally more prone to amplifying hallucinations during the reasoning process compared to their reasoning counterparts. (b) Performance of different models on reasoning and perception tasks in the RH- RH-Bench dataset. Better performing models are positioned in the upper right corner. Baseline non-reasoning models of varying scales typically exhibit weaker reasoning capabilities and fewer hallucination, whereas reasoning models display the opposite trend.
hallucination risk. Moreover, the extension of the reasoning chain further weakens the visual atten- tion allocation, leading increase in hallucinations, as the model becomes more dependent on language-based reasoning rather than visual evidence.
Based on these findings, we further investigate the impact of reasoning chain length on model reasoning and hallucination. The results indicate that the influence of reasoning chain length on bassoing-haalucinatte
exhibits non-monotonic relationship. Additionally, the optimal reasoning range differs across tasks, while traditional evaluation metrics, such as accuracy and hallucination rate, are inadequate for capturing the dynamic balance between reasoning and visual grounding.
To address this, we introduce RH-AUC, new metric designed to assess the balance between reasoning and hallucination in multimodal reasoning models. This metric is computed by calculating the area under the curve formed by reasoning performance and hallucination performance different reasoning lengths, with higher values indicating better balance. Alongside this metric, we release RH-Bench, a diagnostic benchmark containing 1,000 samples across various reasoning and perception tasks, with each task featuring both multiple-choice questions and open-ender questions. Through the evaluation RH- RH--encch, we observe three key findings: (i) Larger models typically demonstrate better reasoning and hallucination balance. (ii) RL-only training models promote more adaptive reasoning, resulting in a better balance between reasoning and hallucination compared to SFT+RL. (ii)) Reasoning- Hallucination balance is more influenced by the types and domains of the training data than by its volume. To sum up, our contributions are listed as follows:
We observe that multimodal reasoning models are more prone to hallucinations than their non- reasoning counterparts perception tasks, which can be attributed a decline in visual attention allocation. Longer reasoning chains further diminish visual attention.
We reveal that the relationship between reasoning chain length and the model's I's reasoning and perception performance is non- non-monotonic, with the optimal length varying across tasks.
We introduce the new RH-AUC metric and the RH--Becc diagnostic dataset to systematically evaluate the balance between reasoning and hallucination across varying reasoning lengths in multimodal reasoning models.
2 Multimodal Reasoning Can Amplify Visual Hallucination
In this section, we begin by investigating whether multimodal reasoning models introduce more hallu- cination in perception-focusee tasks. Specifically, we compare 8 recent multimodal reasoning models against their backbone non-reasoning-bant based counterparts across multiple hallucination benchmarks, including MMVP
42 MMEval-Pro (13), VMCBench Bingo [5).MLLL (39.
2.1 Hallucination Increases Consistently Compared to Base Models
To systematically assess the impact of multimodal reasoning on visual grounding, we evaluated eight reasoning- auggeeted models against their non-reasoning Qwen2.5-VL backbones on five

---

hallucination datasets. As shown in Figure all reasoning models trace markedly smaller radar areas than their baselines, indicating uniformly higher hallucination rates on perception-foeuse focused tasks. This deficit remains consistent at both the 3 B and 7 B scales, demonstrating that the elevated hallucination rate stems from the reasoning paradigm itself rather than model size.
Bingo Score
Bingo Score
1.
MMHALU AA
3.5
MMVP MMHALU
133
MMVP
3.20
3.12
24
756
$45
VMCBeach
MMEEvll-Pro
Bench
MMEEvalPrr
Qwam2.5-L1-388 (Basc)
Carr-REFT
LLM-RI
Quea2.5-VL-78 (Basc)
Vislon-I
ThinkLite-VL.
RI-OneVaisse
OpenVLThinker
Comparison of reasoning and non-reasoning models five perception benchmarks. Results are shown for 3B models (left) and 7B models (right). Higher scores indicate lower hallucination.
Figure 2:
2.2 Does Training Paradigm Matter? Comparison Between RL and SFT+RL
Current multimodal reasoning models typically adopt one of two training regimes: (1) pure re- inforcement learning (RL-only) or (2) supervised fine-tuning followed by reinforcement learning (SFT+RL). Figure shows a consistent performance hierarchy across four perception benchmarks: The Qwen2.5-VV baseline achieves the highest scores, followed by RL-only fine-tuning, with the SFT+RL pipeline performing the worst. This pattern highlights the robustness of baseline model in visual grounding and indicates that subsequent RL or hybrid fine-tuning weakens this robustness, with the supervised-precedt RL strategy leading to the most significant performance degradation.
Base
SFT+RL
MMVP Perception)
MM Perception)
(7B Perception)
MMHALU Perception)
3.6
47:30
$4.50
47-
3.5
3.50
25
Accuracy
Accuracy
24
85
73.50
2.3
44.50
64-
3.2
3.20
21-
11.00
3.1
3.00
Base
RL.
SFT+RI.
Bave
RL.
SFT+RL
Baae
RL.
SFT-RL
Base
RL.
SPT-RL
Figure 3: Performance across four perception benchmarks comparing Base, RL, and SFT+RL.
2.3 Case Study Hallucinations in Reasoning Models
Figure ppresents two representative hallucination patterns observed in multimodal reasoning models, arising from visual misrecognition and reasoning bias, respectively. In Figure the reasoning model fails to identify fine-graine visual cues and miscounts four individuals as three, reflecting localized deficiency in visual perception. Figure the reasoning model increasingly relies on linguistic priors during the reasoning process while overlooking early visual evidence, ultimately generating an incorrect response. In contrast, the baseline model exhibits a lower hallucination rate under identical inputs. These observations raise a crucial question: why do multimodal reasoning models, despite their strong reasoning performance, exhibit weakened visual grounding? the next section, we provides an in-depth analysis based the internal attention mechanisms of the reasoning models.

---

Bona: Yos, the image thoit for peoplt: odgt mone rhilnon, wnaning uettonn
Are there four people
Is there The iniog??
scone showd jucc stand with cranges and und people
ospear This image?
wover
onange jucc prices for orsnge jucc and 100% natural jiice Looking beendi boxss trngg oranges chaleaeand Hewever, since It's jide photl lioe that caze would be present sarning
Reasoning: rttaks the
soove of the seech thrne indiviceess woar vetrutt. One person holding orange cur
stall
thind [- JiTann's no in auliotion caty Toma people beyond thase thrne c/thiiic
thrns indtadhudl
presense samaesss//ieeeemmeeeem wwweeeeeeeeeeeeee.........
context strongly dugger's ricil
ore stean Ansverr
Two common types of hallucination patterns observed in multimodal reasoning models. (a) corresponds hallucinations caused by visual misrecognition, while (b) reflects hallucinations arising from reasoning biases. Hallucinated spans are highlighted in red.
Figure 4:
Takeaway Reasoning Models Amplify Visual Hallucinations
Across training paradigms and model scales, multi-modal reasoning models exhibit consistent drop in accuracy and rise in hallucination rates on general visual benchmarks.
Why Reasoning Models Amplify Hallucinations?
Many previous studies have investigated the role of attention mechanisms in hallucination, identifying insufficient attention allocation as a potential key factor contributing to hallucinations 14 16 53 In this section, conduct attention based analysis explore the underlying amplification multimodal reasoning models. 3 inddiaaes that hallucinations may result from limited attention allocated visual inputs, while Section 3.2 shows that longer reasoning chains further weaken the model's visual focus.
of hallucination
3.1 Hallucination Resulting from Weak Visual Attention
We conduct a comparative analysis of the attention distributions over visual, instruction, and system tokens across all layers the reasoning and non-reasoning models. As shown Figure 55. the reasoning model consistently assigns low attention to visual tokens, with further decrease observed deeper layers, indicating limited ability to integrate visual evidence. Meanwhile, more attention is shifted to instruction tokens, reflecting heightened reliance linguistic priors. In contrast, the non non-reasoning maintains relatively high and stable level of visual attention from shallow to intermediate layers. The visual attention heatmap in Figure 5p further supports this observation: while the non-reasoning model progressively focuses on semantically salient regions, the reasoning model exhibits sparse and dispersed attention, failing to consistently engage with key visual areas. This phenomenon indicates that the weakening of visual attention undermines the reasoning model's ability to achieve effective visual grounding, exacerbating the occurrence of hallucinations.
Instruction
Takens
Is there cheese on the posto?
Qwen2.5-V D-nn2.5-VL-TDD (Non-Reasunin Ressmingg
RI-OneVision (Ravening)
Layer
Layer10
Layer15
1.0
1.1
Quxen2.5 S-V1--------
Aftenten
0.6
RI VneVissio
0.2
Layer
(a)
(b)
Figure 5: Attention allocation and visual grounding between reasoning and non reasoning models.
The reduction of visual attention in reasoning models amplifies visual hallucinations.

---

3.2 Visual Focus Declines with Longer Reasoning Chains
As shown in Figure we visualize the attention distributions of the reasoning model under two reasoning modes: normal thinking and overthinking, As the reasoning chain length increases, the heatmaps clearly reveal systematic shift in the model's attention focus: under the overthinking mode, attention to visual tokens significantly decreases, while attention to instruction tokens intensifies. This pattern indicates that longer reasoning chains cause the model to increasingly rely on linguistic cues rather than grounded visual evidence. For instance, when asked whether gray wall is present, the model under normal thinking correctly identifies the gray well and provides a correct response. In contrast, under over-reasoning conditions, the model exhibits further diminished attention visual tokens, with increased focus directed toward the end the user instruction. This suggests that longer reasoning chains tend to further exacerbate the degradation of the model's visual grounding, potentially leading to an increase in hallucinations.
Normal Thinking
Overthinking
groy the image?
Image
Image
Figure 6: Attention shift the reasoning model under different reasoning length. normal thinking. the model generates outputs as typically expected, while in overthinking, the reasoning length is adjusted using Latent State Steering (Section[4.]. I.T). Longer reasoning chains further exacerbate the degradation of attention visual information and focus toward linguistic priors.
Effects of Reasoning Length on Reasoning-Halluciienn
Balance
In this section, explore the impact of reasoning length the balance between hallucination and reasoning. Section provides an overview of the proposed control strategy: latent state steering as well as techniques that are previously used in the literature budget forcing, and test time scaling. In Section 4.2 we explore the optimal generation length for various benchmarks and analyze the trade-off between hallucination and reasoning performance reasoning length varies.
4.1 Overview of Reasoning Length Control Straregies
To systematically control the reasoning length in reasoning models, we adopt three strategies:
(I) Token Budget Forcing: A hard constraint on reasoning length is enforced by predefining a generation budget at decoding time, directly limiting the number of tokens allocated for the reasoning.
(2) Test Time Scaling: Reasoning is incrementally extended during inference through staged gener- ation. The model first produces partial reasoning under a 4096-token constraint and halts midway. then prompted to continue by appending a simple token ("Wait"), enabling soft extension of reasoning while preserving contextual coherence.
(3) Latent State Steering: Inspired by recent works on latent space steering for behavior control in large language models [23] 22, I 229], we propose a method to steer the model toward generating reasoning traces of varying lengths. Specifically, we extract steering directions from the post-attention hidden states by computing the difference of latent states between long and short reasoning trajectories. These direction vectors are obtained and applied across all layers of the text decoder, with scaling factor controlling both the magnitude of guidance on the reasoning length. Specifically, we collect responses from the test benchmark and categorize them into long reasoning traces Riong and short reasoning traces Rahor based on token length. The query and reasoning steps for each sample are input into the model, from which hidden representations S are extracted each layer. S (g. 1) denotes the hidden representation layer e for token position t in the response to query q. We
5

---

compute the average hidden representation over reasoning tokens, where Hi represents the set of token positions within the reasoning span. The average representation is then calculated across the long and short reasoning traces to obtain layerwise embeddings:
Song
Σ (q, IEH.
Shoot
Σ (q. KEH.
(I)
RRong
HH
RRhoort
q
The reasoning length direction at layer E is defined the difference between the long and short embeddings, denoted as d', which captures the variation in the model's representation resulting from different reasoning chain lengths. To adjust the hidden representation based this direction, We introduce a parameter a E 1-0.15,0.0 0.15] to dynamically control the reasoning length and its magnitude. As increases, the length of the reasoning chain extends, as shown below:
d' =
S short
Satering
S add.
These strategies are applied to five representative multimodal reasoning models and evaluated on six benchmark datasets, covering both reasoning and perception tasks. In Figure 7 we present two benchmarks for both tasks. All implementation details and results are provided in Appendix C.
R1-One Visloon
TtinkLite-LL
Occan-Ri
Open
MM-Fureka
Vata (Reasoning)
MathVision (Reasoning)
MMMaau (Halllcination)
MMVP (Hallucination)
70:0
Accurscy (%2
Hallucination
12:0
65.
25.0
45
00.0
22.5
35.0
439 Thinking Length
300
500
5E0
600
253
300
90
1.00
150
Thinking Length
lhinking Length
Thinking Length
Figure 7: Reasoningg-Hallucciattoo balance of multimodal reasoning models under varying reasoning lengths. Thinking lengths are controlled within [0-600] tokens for reasoning and [0-300] for hallucination, corresponding to the longer chains required for reasoning and shorter for hallucination.
4.2 Dynamic Balance Between Reasoning and Hallucination
Non-monotonic Effect of Reasoning Length Reasoning and Perception Performance. The relationship between reasoning length and model performance typically exhibits a non-monotonic pattern under reasoning and perception tasks. Across various length control strategies, a consistent trend emerges: moderate reasoning depth tends to yield optimal performance, whereas overly short or excessively long reasoning chains often lead to decline in accuracy. As shown in Figure 7 we employ the Latent State Steering method adjusts the thinking step for reasoning and perception tasks. evident that as the thinking length increases, the model's performance across tasks generally follows a rising-then- thiingtthen-fallngg falling trajectory. This indicates that enhanced reasoning does not linearly improve model performance, but instead follows dynamic trade-off pattern.
Task-Specific Variability of Optimal Reasoning Intervals. While most tasks exhibit non- monoooooo relationships between reasoning length and performance, further observe that the optimal rea- soning length varies significantly across tasks. Figure reeveas that reasoning benchmarks such as Math (27) tend benefit from longer reasoning chains, whereas perception and hallucination- oriented tasks such as MMHalu achieve their best performance at shorter or moderate lengths. This indicates that the balance between reasoning depth and performance is sascisiec,,i,, and unified length control strategies are unlikely to be effective across all task types.
Impact of the Zerothink Condition. Zerothink retains the reasoning structure but lacks substantive content. As shown in Figure 7, this setting leads to a consistent drop in model performance on both reasoning and perception benchmarks, notably lower than results under normal reasoning lengths. These results indicate that the absence of reasoning content diminishes the reasoning model's performance in both perception and reasoning.
Limitation of Conventional Metric. Conventional metries like reasoning accuracy and hallucination rate, when computed at fixed generation length, fail to capture the dynamic balance between deeper

---

reasoning and perception. Figure T reasoning lengths, making misleading to evaluate models using single-point metrics or simple averages between reasoning and hallucination performance. For instance, short reasoning trace may yield a lower hallucination rate but poor reasoning depth, while longer trace may improve reasoning at the cost of increased hallucination, yet both scenarios could yield the same average score.
shows that reasoning and perception often peak at different
To capture this evolving balance, in the next section, we propose an AUC-style metric that summarizes the balance curve between reasoning and perception fidelity across various reasoning lengths. This provides more faithful and holistic measure of performance, revealing both the model's optimal balance and its stability across varying generation lengths.
Takeaway 2: Moderate Reasoning Length Strikes the Best Reasoning- Gaasoning-HHllucinnn Hallucination Balance
Reasoning length exerts a non-monotonic effect on model performance: both insufficient and excessive reasoning degrade accuracy, and the optimal length is task-dependent.
Evaluation on the
Balance
To comprehensively quantify the balance between reasoning and hallucination in multimodal large reasoning models at different reasoning depths, we introduce a new metric RH-AUC. This metric captures how hallucination risk evolves with reasoning depth while also reflecting the cumulative effects of reasoning and perception. Additionally, we present RH-BBench, a new diagnostic dataset of 1000 samples, designed for the integrated evaluation of reasoning and perception tasks. offering a robust basis for analyzing reasoning ability and perceptual hallucinations.
5.1 Setup
Benchmark Overview. RH-Bench consists of two types of tasks: reasoning and perception, with each task including two types of questions: multiple-choice and open-ended. The reasoning task includes 500 samples sourced from Vision [44], Math Vista (27). MMMU (55), and ScienceQA (28), while the visual perception task includes 500 samples from MMhalu, MMVP, HallusionBench, and VMCBench. Both task types use accuracy the evaluation metric. For multiple-choice questions, evaluation is based on matching the final options. For open-ended questions, both tasks are evaluated using GPT-4o. The reasoning task determines whether the generated response is consistent with the correct answer, whereas the visual task evaluates the generated response against the correct answer, assigning score within the range of 0 to 6. Responses with score below 3 are classified as hallucinations. All sample ground-truth and evaluation answers have undergone manual inspection.
Method
Paradigms
Perception
Reasoning
Training Data
RH-AUC
Acc.(%)
Length
Acc.(%)
Length
Perc.
Reas.
LLM-R1-3B
RL
48.7
121.9
43.8
391.8
65k
40k
0.46
Cum-ReFFT-3B
SFT+RL
50.6
133.7
42.5
472.61
6k
3k
0.47
Ocean-R1-35
52.8
131.2
45.6
414.5
63k
0.53
RI RI-OneVision-75
SFT+RL
55.7
162.9
44.2
457.3
80K
77k
0.46
ThinkLite-VL-77
RL
63.3
110.4
50.4
435.4
62k
8k
0,52
OpenVLThinner-77B
SFT+RL
59.2
187.7
48.9
460.1
25k
25k
0.54
MM-Eurcka-7B
RL
62.0
139.6
54.0
450.5
15k
0.55
MM-R 1-7B
RL.
60.3
139.6
54.0
430.0
6k
0.57
Ocean-R1- R1-BB
RL
62.3
90.4
51.8
262.2
20k
63k
0.63
Table 1: Comparison of model performance on RH-Bench, including task-specific accuracy and RH- AUC scores. Perc. and Reas. respectively denote training data for visual perception and reasoning.
RH-AUC We define reasoning length as T, which controls the extent of the model's generated reasoning trace. For each length T. compute Ry, which represents the reasoning performance length T, and Hr, representing performance on hallucination at the same length.
By evaluating the model multiple lengths on the RfI-bench benchmark, we obtain series of (Rr, Hr) pairs that form balance curve between reasoning and perceptual hallucination. To compute the area under this curve, we first sort the pairs in ascending order of reasoning performance Rr. Let
7

---

the sorted indices be denoted as "O,,,"..... To ensure comparability across models, both Ry and Hr are min-max normalized to the range [0. 1]. The RH-AUC is then computed using the trapezoidal rule as:
such that Ryo: S RTO S ... S Ry
R
RTo (Hyo44 + Hy()),
RH-AUC-
(3)
1=0
where n is the number of evaluated reasoning lengths. A higher RH-AUC indicates model that better balances reasoning and hallucination across different reasoning lengths.
SET ILL
RL
shot omage untled
(RL))
Leyes skues
Choccs (A) Fee Lareed
shoe appare in Therll the dhen it
RRI Reasu- ACC (N)
Aperago MILALC
sanng tidd ana drraraeen
Legil kakess
Openstnniker (5F (STTHL)
The trage sheve aerpee tylng
the injong
Hoat the then mecess bor infermet that the shoe ciricily fert
Low-Tlick
Sarmal-Thick
Marr-TBiak
SFT+RI (3888. (1B)
(c)
Figure 8: (a) Accuracy trends on the RII-Bench reasoning task across different reasoning lengths for 3B and 7B models. Larger models typically exhibit more stable performance across varying reasoning lengths. (b) Comparison of SFT+RL and RL-only training paradigms in terms of RII-AUC, with arrow directions indicating the increase in reasoning length for SFT+RL relative to RL-only. RL-only training tends to generate more concise reasoning chains, leading to a better perception hallucination balance. (c) Case study comparing RL-only and SFT+RL models. SFT+RL models often introduce rigid imitation reasoning paths, which limit the flexibility of visual reasoning.
5.2 Result Diagnosis
We conduct in-depth analysis of model performance based on the evaluation results from the RH-Bench diagnostic dataset, investigating the influence of three key factors: model scale, training paradigm, and training dataset on the massoning-hallucinatii balance.
Model Scaling, As shown in Table D the 7B model generally outperforms the 3B model in RH-Bench demonstrating higher RH-AUC, primarily due to its larger parameter size and stronger representational capacity. As illustrated in Figure Sh. the larger model maintains higher stability, especially under longer reasoning chains, while the smaller models show a noticeable decline in performance. This suggests that larger models typically exhibit better robustness and adaptability.
Training Paradigms. A comparison between the two-stage SFT+RL-trained model and the RL-only trained model reveals that RL demonstrates stronger balance between reasoning and perception. For example, in Figure L although the Open VLThinker model maintains a longer reasoning chain, the introduction of redundant reasoning interferes with visual perception, leading to an incorrect inference that the shoe is already tied. In contrast, the RL-only model, Ocean-R1 11, uses a shorter reasoning chain, enabling to more efficiently capture key visual features and avoid unnecessary complex reasoning steps. This advantage is particularly evident at different reasoning length, as shown in the average RH-AUC in Figure Ep, which significantly higher for the RL-onnl model compared to SFT+RL. This phenomenon suggests that although SFT helps the model learn reasoning formats, it may introduce rigid imitation reasoning paths, limiting the model's adaptability to dynamic tasks and ultimately resulting in redundant reasoning. In contrast, RL encourages the model to generate more adaptive reasoning behaviors, enhancing the integration of reasoning and perception.
Training Dataset. The diversity and quality of training data play a crucial role in the reasoning- hallucination balance of models. Through a statistical analysis of the multimodel reasoning models training data and a comparison with the results in Table [ we have observed some interesting phenomena: (1) More visual perception data does not necessarily improve the balance between reasoning and perception. Increasing the training samples of visual perception data can enhance the balance of reasoning models to a certain degree. For example, the Think Lite- VI. model, supported by large scale visual perception data, demonstrates strong hallucination and reasoning balance. Similarly,
8

---

the Ocean-RI model adopts a two-stage training strategy, first enhancing reasoning ability and then strengthening visual perception, achieving the highest RH-AUC on RH-bench. However, this phenomenon is not consistent, For example, despite the R1-One Vision model utilizing large amount of visual perception data, it demonstrates a weaker balance between reasoning and perception, which may be attributed to the limitations of its training paradigm design. (2) Perception and Reasoning balance can be achieved through training on domain-specific data. Training on domain-specinee data helps enhance the balance of the reasoning model. For example, the MM-Eureka model, trained on a larger mathematical dataset, shows a higher RII-AUC, proving its effectiveness in balancing reasoning and perception. Similarly, despite being trained on only 6k mathematical data, the MM-R1 model still performs well on RH-bench. This highlights the potential of domain-speciff data to stimulate the balance capabilities of reasoning models, even with smaller datasets. (3) The size of the training data is not always a guarantee for the reasoning- size does not always directly correlate with the model's balance capability. For example, both the LLM-R1, 1, trained on over 60k visual perception samples, and the R1 R1-One Vision, with a dataset of 150k samples, exhibit inadequate rasoning-halllciing balance, with the RH-AUC of only 0.46.
balance. The traing data
6 Related Work
Multimodal Reasoning Tasks. Multimodal reasoning requires integrating information across modalities to solve complex problems. is generally categorized into general reasoning and domain- specific reasoning. General reasoning typically occurs in natural image scenarios, where models must combine visual perception with knowledge and commonsense. Representative benchmarks include multiple-choice datasets such MMMU (55] MMVP 422, MMBench [24], MMStar K}
MMEval-Pro [3], and VMCBench [61], as well as open-ended evaluations like Bingo (5) MMHAL-
Bench 13 POPE [9). CHAIR R5 and HallusionBench 112] Domain-specific reasoning focuses on technical tasks within particular domains. For mathematical reasoning, benchmarks such as Math (27). MATH-Vision Vision [44), MM-Math (37). WeMath (34) evaluate models' ability to solve math problems grounded in visual contexts. For physical reasoning, datasets like PhysBench [4] and test understanding of physics and commonsense reasoning from visual inputs.
CRAVE (38
Reinforcement Learning in MLLMs. Recent approaches enhance the reasoning capabilities of multi- modal large models by incorporating chain-of-thouge supervision during supervised fine-tuming or re- inforcement learning 62 58 41 50 [2]. Methods like RLHF-V (54), MaLaVA--Rasoone [60], and
Insight-V [|] leverage scage Cty-style datasets and preference optimization to improve model reasoning. Following DeepSeek-RR R1, the GRPO (Group Relative Policy Optimization) algorithm has become standard paradigm in training multimodal large reasoning models [26 57 47 43 18 45 25). Some models, such as RI-On RI-One Vision [51], Reason-REF [40] and R1-VL [59]. follow a two-stage SFT RL pipeline, while others like Ocean-RI (20]. ThinkLite- VL 446. and MM-Eureka [30 apply rule-based reinforcement learning directly at scale.
Conclusion
In conclusion, this paper investigates the balance between reasoning and hallucination in multimodal reasoning models, with a focus on how reasoning chain length and visual attention allocation impact performance. While longer reasoning chains enhance performance on complex tasks, they also exacerbate hallucinations by diminishing visual attention and increasing reliance on language priors. To address these challenges, the paper introduces the RH-AUC metric and the RH-Bench benchmark, which provide a systematic method to evaluate the balance between reasoning ability and hallucination risk. The findings reveal that aasooninnn-agmmneeee models are more prone to hallucinations, highlighting the importance of developing evaluation frameworks that assess both the quality of reasoning and the accuracy of perception.
Limitation. Although our study provides a comprehensive analysis of visual hallucinations in multimodal reasoning models, it also has several limitations. First, our evaluation is limited to models built on the Qwen2.5-V6 backbone, which may constrain the generalizability of our findings to architectures with different modalities influence of training data is based solely on technical reports and publicly available documentation of existing models, without conducting controlled retraining experiments. Therefore, our conclusions are observational and may not fully capture causal effects.
pretraining objectives. Second, our analysis of the

---

References
[1] Zouying Cao, Yifei Yang, and Hai Zhao. Nothing in excess: Mitigating the exaggerated safety for Ilms via safety- cefesy-conscious activation steering. arXiv preprint arXiv:2408. 1491, 2024.
[2] Edward Y. Chang, Yuxuan Tong, Morry Niu, Graham Neubig, and Xiang Yue. Demystifying long chain thought reasoning in llms. CoRR, abs/2502.03373, 2025.
[3] Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang. Yu Qiao, Dahua Lin, al. Are we on the right way for evaluating large -aagugge 2024.
models? arXiv preprint 2rXiv:2403.200
[4] Wei Chow, Jiageng Mao, Boyi Li, Daniel Seita, Vitor Guizilini, and Yue Wang. Physbench: Benchmarking and enhancing vision- Saiiunla models for physical world understanding. arXiv preprint arXiv:2501.16411 2025.
[5] Chenhang Cui, Yiyang Zhou, Xinyu Yang, Shirley Wu, Linjun Zhang, James Zou, and Huaxiu Yao. Holistic analysis of hallucination gpt-4v (ision): Bias and interference challenges. arXiv preprint arXiv:2311.03287 2023.
[6] DeepSeek-Ar AI, Daya Guo. Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma. Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang. Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Han Bao, Hanwei Xu, Haocheng Wang, Honghui Ding, Huajian Xin, Huazuo Gao. Hui Qu, Hui Li, Jianzhong Guo, Jiashi Li, Jiawei Wang. Jingehang Chen, Jingyang Yuan, Junjie Qiu, Junlong Li, J. L. Cai, Jiaqi Ni, Jian Liang, Jin Chen, Kai Dong, Kai Hu, Kaige Gao. Kang Guan, Kexin Huang, Kuai Yu, Lean Wang, Lecong Zhang, Liang Zhao, Litong Wang. Liyue Zhang, Lei Xu, Leyi Xia, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Meng Li. Miaojun Wang. Mingming Li, Ning Tian, Panpan Huang, Peng Zhang, Qiancheng Wang. Qinyu Chen, Qiushi Du Ruiqi Ge, Ruisong Zhang, Ruizhe Pan, Runji Wang, R. J. Chen, R. L. Jin, Ruyi Chen, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shengfeng Ye, Shiyu Wang, Shuiping Yu, Shunfeng Zhou, Shuting Pan, and S. S. Li. Deepseek-rl: Incentivizing reasoning capability in llms via reinforcement learning. CoRR, abs/2501. 12948, 2025.
[7] DeepSeck-AI, Aixin Liu, Bei Feng. Bing Xue, Bingxuan Wang. Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangho Hao, Guanting Chen, Guowei Li, H. Zhang, Han Bao, Hanwei Xu, Haocheng Wang. Haowei Zhang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Li, Hui Qu, L. Cai, Jian Liang, Jianzhong Guo, Jiaqi Ni, Jiashi Li. Jiawei Wang, Jin Chen, Jingchang Chen, Jingyang Yuan, Junjie Qiu, Junlong Li, Junxiao Song, Kai Dong, Kai Hu, Kaige Gao, Kang Guan, Kexin Huang, Kuai Yu, Lean Wang, Lecong Zhang, Lei Xu, Leyi Xia, Liang Zhao, Litong Wang, Liyue Zhang, Meng Li, Miaojun Wang, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Mingming Li, Ning Tian, Panpan Huang, Peiyi Wang, Peng Zhang, Qiancheng Wang. Qihao Zhu, Qinyu Chen, Qiushi Du, R. J. Chen, R. L. Jin, Ruiqi Ge, Ruisong Zhang, Ruizhe Pan, Runji Wang. Runxin Xu, Ruoyu Zhang, Ruyi Chen, S. Li, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shaoqing Wu, Shengfeng Ye, Shengfeng Ye, Shirong Ma, Shiyu Wang, Shuang Zhou, Shuiping Yu, Shunfeng Zhou, Shuting Pan, T. Wang, Tao Yun, Tian Pei, Tianyu Sun, W. L. Xiao, and Wangding Zeng. Deepseek-v3 v3 technical report. CoRR, abs/2412. 19437, 2024.
[8] Huilin Deng, Ding Zou, Rui Ma, Hongchen Luo, Yang Cao, and Yu Kang. Boosting the generalization and reasoning of vision language models with curriculum reinforcement learning, 2025.
[9] Yihe Deng, Hritik Bansal, Fan Yin, Nanyun Peng, Wei Wang, and Kai-Wei Chang. Open- vlthinker: early exploration complex vasion-language reasoning via iterative self- improvement, 2025.
10

---

[10] Yuhao Dong, Zuyan Liu, Hai-Long Sun, Jingkang Yang, Winston Hu, Yongming Rao, and Ziwei Liu. Insight-v: Exploring long-chain visual reasoning with multimodal large language models. arXiv preprint arXiv:241 1. 14432, 2024.
[11] Yuhao Dong, Zuyan Liu, Hai-Long Sun, Jingkang Yang, Winston Hu, Yongming Rao, and Ziwei Liu. Insight-v: Exploring long-chain visual reasoning with multimodal large language models. arXiv preprint arXiv:2411.144: 2024.
[12] Tianrui Guan, Fuxiao Liu, Xiyang Wu, Ruiqi Xian, Zongxia Li, Xiaoyu Liu, Xijun Wang, Lichang Chen, Furong Huang, Yaser Yacoob, et al. Hallusionbench: an advanced diagnostic suite for entangled language hallucination and visual illusion in large vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14375 14385, 2024.
[13] Jinsheng Huang, Liang Chen, Taian Guo, Fu Zeng, Yusheng Zhao, Bohan Wu, Ye Yuan, Haozhe Zhao, Zhihui Guo, Yichi Zhang, et al. Mmevalpro: Calibrating multimodal benchmarks towards trustworthy and efficient evaluation. arXiv preprint arXiv:2407.00466 2024.
[14] Qidong Huang, Xiaoyi Dong, Pan Zhang, Bin Wang, Conghui He, Jiaqi Wang, Dahua Lin, Weiming Zhang, and Nenghai Yu. Opera: Alleviating hallucination in multi-modal large language models via over-trust penalty and retrospection-allocation In Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition, pages 13418-13422 2024.
[15] Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao, Zheyu Ye, Fei Zhao, Yao Hu, and Shaohui Lin. Vision-rl: Incentivizing reasoning capability in multimodal large language models. arXiv preprint arXiv:2503.06749 2025.
[16] Fushuo Huo, Wenchao Xu, Zhong Zhang, Haozhao Wang, Zhicheng Chen, and Peilin Zhao. Self-introspective decoding: Alleviating hallucinations for large vision-language models. In The Thirteenth International Conference on Learning Representations, 2025.
[17] Sicong Leng, Jing Wang, Jiaxi Li, Hao Zhang, Zhiqiang Hu, Boqiang Zhang, Hang Zhang, Yuming Jiang, Xin Li, Deli Zhao, Fan Wang, Yu Rong. Aixin Sun, and Shijian Lu. Mmrl: Advancing the frontiers of multimodal reasoning. https://githhb.cco comptthub.cmm/Lnnnnee 2025.
[18] Weiqi Li, Xuanyu Zhang, Shijie Zhao, Yabin Zhang, Junlin Li, Li Zhang, and Jian Zhang, Q-insight: Understanding image quality via visual reinforcement learning. arXiv preprint arXiv:2503.22679 2025.
[19] Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Xin Zhao, and Ji-Rong Wen. Evaluating object hallucination in large vision-language models. Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural Lan- guage Processing, pages 292-305, Singapore, December 2023. Association for Computational Linguisties.
[20] Ming Lingfeng, Li Yadong, Chen Song, Xu Jianhua, Zhou Zenan, and Chen Weipeng. Ocean-rl: An open and generalizable large vision- language model enhanced by reinforcement learning. https://gt thub. com/ com/VLM RLMMRLLDcean-- R1 2025. Accessed: 2025-04-03.
[21] Chengzhi Liu, Yuzhe Yang, Kaiwen Zhou, Zhen Zhang, Yue Fan, Yannan Xie, Peng Qi, and Xin Eric Wang. Presenting a paper is an art: Self eelfovmmpove aesthetic agents for academic presentations, 2025.
[22] Sheng Liu, Haotian Ye, Lei Xing, and James Zou. In-context vectors: Making in context learning more effective and controllable through latent space steering. arXiv preprint arXiv22311.06668, 2023.
[23] Sheng Liu, Haotian Ye, and James Zou. Reducing hallucinations in large vision-language models via latent space steering. The Thirteenth International Conference on Learning Representations.
11

---

[24] Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang. Conghui He, Ziwei Liu, al. Mmbench: your multi-modal model an all-around player? In European conference computer vision, pages 216-233. Springer, 2024.
[25] Yue Liu, Shengfang Zhai, Mingzhe Du, Yulin Chen, Tri Cao, Hongcheng Gao, Cheng Wang, Xinfeng Li. Kun Wang, Junfeng Fang, Jiaheng Zhang. and Bryan Hooi. Guardreasoner-vl: Safeguarding vims via reinforced reasoning. arXiv preprint arXiv:2505. 1049, 2025.
[26] Ziyu Liu, Zeyi Sun, Yuhang Zang, Xiaoyi Dong, Yuhang Cao. Haodong Duan. Dahua Lin, and Jiaqi Wang. Visual-rft: Visual reinforcement fine-tuning, arXiv preprint urXiv:2503.01785 2025.
[27] Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. In International Conference on Learning Representations (ICLR), 2024.
[28] Pan Lu, Swaroop Mishra, Tony Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. Learn to explain: Multimodal reasoning via thought chains for science question answering, 2022.
[29] Jinqi Luo, Tianjiao Ding, Kwan Ho Ryan Chan, Darshan Thaker, Aditya Chattopadhyay, Chris Kallison-Burchh and René Vidal. Pace: Parsimonious concept engineering for large language models. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
[30] Fanqing Meng, Lingxiao Du, Zongkai Liu, Zhixiang Zhou, Quanfeng Lu, Daocheng Fu, Botian Shi, Wenhai Wang, Junjun He, Kaipeng Zhang, et al. Mm-eureka: Exploring visual aha moment with rule-based large-scale reinforcement learning, arXiv preprint arXiv:2503.07365. 2025.
[31] Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. sl: Simple test-time scaling, 2025.
[32]
OpenAl. Learning to reason with LLMs. 2024.
[33] Yingzhe Peng, Gongrui Zhang, Miaosen Zhang, Zhiyuan You, Jie Liu, Qipeng Zhu, Kai Yang. Xingzhong Xu, Xin Geng, and Xu Yang. Lmm-rl: Empowering 3b Imms with strong reasoning abilities through two-stage rule-based rl. arXiv preprint wxXXv:2503.07536, 2025.
[34] Runqi Qiao, Qiuna Tan, Guanting Dong, Minhui Wu, Chong Sun, Xiaoshuai Song, Zhuoma Gongque, Shanglin Lei, Zhe Wei, Miaoxuan Zhang, Runfeng Qiao, Yifan Zhang, Xiao Zong, Yida Xu, Muxi Diao, Zhimin Bao, Chen Li, and Honggang Zhang. We-math: Does your large multimodal model achieve human-like mathematical reasoning? CoRR, abs/2407.01284, 2024.
[35] Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns, Trevor Darrell, and Kate Saenko. Object hallucination in image captioning. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 4035-4045, 2018
[36] Haozhan Shen, Zilun Zhang, Qianqian Zhang, Ruochen Xu, and Tiancheng Zhao. Vlm-rl: A stable and generalizable rl rl-style large vision-language model. 2025. Accessed: 2025-02-15, 15.
[37] Kai Sun, Yushi Bai, Ji Qi, Lei Hou, and Juanzi Li. MM-MATH: Advancing multimodal math evaluation with process evaluation and fine-grained classification. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Findings of the Association for Computational Linguistics: EMNLP 2024, pages 1358-1375, Miami, Florida, USA, November 2024. Association for Computational Linguistics.
[38] Shangkun Sun, Xiaoyu Liang, Bowen Qu, and Wei Gao. Content-rich aige video qual- ity assessment via intricate text alignment and motion-aware consistency. arXiv preprint 20Xiv2250..0407, 2025.
12

---

[39] Zhiqing Sun, Sheng Shen, Shengcao Cao, Haotian Liu, Chunyuan Li, Yikang Shen, Chuang Gan, Liang- Yan Gui, Yu-Xiong Wang, Yiming Yang, et al. Aligning large multimodal models with factually augmented rlhf. arXiv preprini arXiv:2309. /4525, 2023.
[40] Huajie Tan, Yuheng Ji, Xiaoshuai Hao, Minglan Lin, Pengwei Wang, Zhongyuan Wang, and Shanghang Zhang. Reason-rfi: Reinforcement fine-tuning for visual reasoning. arXiv preprint urXiv:2503.20752 2025.
[41] Omkar Thawakar, Dinura Dissanayake, Ketan More, Ritesh Thawkar, Ahmed Heakl, Noor Ahsan, Yuhao Li, Mohammed Zumri, Jean Lahoud, Rao Muhammad Anwer, et al. Llamav-ol: Rethinking step-by-step visual reasoning in llms, arXiv preprint arXiv:2501.06186 2025.
[42] Shenghang Tong, Zhuang Liu, Yuexiang Zhai, Yi Ma, Yann LeCun, and Saining Xie. Eyes wide shut? exploring the visual shortcomings of multimodal 1lms. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 9568-9578 2024.
[43] Haozhe Wang, Chao Qu Zuming Huang, Wei Chu, Fangzhen Lin, and Wenhu Chen. VI- rethinker: Incentivizing self-reflection of rision-aaguuae models with reinforcement learning. arXiv preprint arXiv:2504.08837 2025.
[44] Ke Wang, Junting Pan, Weikang Shi. Zimu Lu, Houxing Ren, Aojun Zhou, Mingjie Zhan, and Hongsheng Li. Mcasuring multimodal mathematical reasoning with math- vision dataset. In The eiiht Conference Neural Information Processing Systems Datasets and Benchmarks Track, 2024.
[45] Weiyun Wang, Zhangwei Gao, Lianjie Chen, Zhe Chen, Jinguo Zhu, Xiangyu Zhao, Yangzhou Liu, Yue Cao, Shenglong Ye, Xizhou Zhu, et al. Visualprm: An effective process reward model for multimodal reasoning. arXiv preprint arXiv:2503. 10291, 2025.
[46] Xiyao Wang, Zhengyuan Yang, Chao Feng, Hongjin Lu, Linjie Li, Chung- Shing-Chigg& Lin, Kevin Lin, Furong Huang, and Lijuan Wang. Sota with less: Mcts- dets-guided sample selection for data-efficient visual reasoning eeff-improvement arXiv preprini arXiv:2504.0P9 07934, 2025.
[47] Wenyi Xiao, Leilei Gan, Weilong Dai, Wanggui He, Ziwei Huang, Haoyuan Li, Fangxun Shu, Zhelun Yu, Peng Zhang, Hao Jiang, et al. Fast-slow thinking for large vision-language model reasoning, arXiv preprint arXiv:2504.1844 2025.
[48] Guowei Xu, Peng Jin, Li Hao, Yibing Song, Lichao Sun, and Li Yuan. Llava-o1: Let vision language models reason step-by-step. arXiv preprint arXiv:2411.10440 2024.
[49] Guowei Xu, Peng Jin, Hao Li, Yibing Song, Lichao Sun, and Li Yuan. Llava-cot: Let vision language models reason step-by-step. 2024.
[50] Haotian Xu, Xing Wu, Weinong Wang, Zhongzhi Li, Da Zheng, Boyuan Chen, Yi Hu, Shijia Kang, Jiaming Ji, Yingying Zhang, et al. Redstar: Does scaling long-cot data unlock better slow-reasoning systems? arXiv preprint arXiv:2501.11284 2025.
[51] Yi Yang, Xiaoxuan He, Hongkun Pan, Xiyan Jiang, Yan Deng, Xingtao Yang. Haoyu Lu, Dacheng Yin, Fengyun Rao, Minfeng Zhu, Bo Zhang, and Wei Chen. Rl-onevision: Ad- vancing generalized multimodal reasoning through cross-modal formalization. arXiv preprint arXiv:2503. 10615, 2025.
[52] Yuzhe Yang, Yifei Zhang, Minghao Wu, Kaidi Zhang, Yunmiao Zhang, Honghai Yu, Yan Hu, and Benyou Wang. Twinmarket: A scalable behavioral and social simulation for financial markets, 2025.
[53] Hao Yin, Guangzong Si, and Zilei Wang. Clearsight: hallucination mitigation in multimodal large language models. arXiv preprint (xX.xx::5033.110000 2025.
Visual signal enhancement for object
[54] Tianyu Yu, Yuan Yao, Haoye Zhang, Taiwen He, Yifeng Han, Ganqu Cui, Jinyi Hu, Zhiyuan Liu, Tai-Tao Zheng, Maosong Sun, et al. Rlhf-v: Towards trustworthy milms via behavior alignment from fine-grained correctional human feedback. In Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition, pages 13807-13816, 2024.
13

---

[55] Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, al. Mmmu: massive discipliii multimodal understanding and reasoning benchmark for expert agi. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pages 9556 9567, 2024.
[56] Liu Yuqi, Peng Bohao, Zhong Zhisheng, Yue Zihao, Lu Fanbin, Yu Bei, and Jia Jiaya. zerozero: Reasoning-chain guided segmentation via cognitive reinforcement, 2025.
[57] Liu Yuqi, Peng Bohao, Zhong Zhisheng, Yue Zihao, Lu Fanbin, Yu Bei, and Jia Jiaya. Seg-zero: Reasoning-chain guided segmentation via cognitive reinforcement, 2025.
[58] Di Zhang. Jianbo Wu, Jingdi Lei, Tong Che, Jiatong Li, Tong Xie, Xiaoshui Huang, Shufei Zhang, Marco Pavone, Yuqiang Li, al. Llama-berry: Pairwise optimization for ol-like olympiad-level mathematical reasoning. arXiv preprint arXiv:2410.02884, 2024.
[59] Jingyi Zhang, Jiaxing Huang, Huanjin Yao, Shunyu Liu, Xikun Zhang, Shijian Lu, and Dacheng Tao. R1-vI: Learning to reason with multimodal large language models via step-wise group relative policy optimization. arXiv preprint arXiv:2503. 2025.
[60] Ruohong Zhang, Bowen Zhang, Yanghao Li, Haotian Zhang, Zhiqing Sun, Zhe Gan, Yinfei Yang, Ruoming Pang, and Yiming Yang. Improve vision language model chain-of-thought reasoning. arXiv preprint 1rx1:22400.16/96 2024.
[61] Yuhui Zhang, Yuchang Su, Yiming Liu, Xiaohan Wang, James Burgess, Elaine Sui, Chenyu Wang, Josiah Aklilu, Alejandro Lozano, Anjiang Wei, et al. Automated generation of chal- lenging multiple-choice questions for vision language model evaluation. arXiv preprint arXiv:2501.03225, 2025.
[62] Yu Zhao, Huifeng Yin, Bo Zeng, Hao Wang. Tianqi Shi, Chenyang Lyu, Longyue Wang, Weihua Luo, and Kaifu Zhang. Marco-ol: Towards open reasoning models for open-ended solutions. arXiv preprint 1 14405, 2024.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
Our contributions are summarized in the Abstract and Introduction sections.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Please see Conclusion.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
15

---

Justification:
This paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Please see the Setup sec.5.1 and Detailed Experimental Settings in appendix A
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
16

---

Answer: [Yes]
Justification:
As mentioned in the Abstract, our code and data will be made publicly
available.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cm ec/
for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https for more details.
//nips. //nips.c//pil wwcppbblc/guuiicuccccccc
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
The detailed results can be found in Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer:
[Yes]
Justification:
The detailed results can be found in Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
17

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
The detailed results can be found in Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics http:: httppp////.aerr...cco//////////a
Answer: [Yes]
Justification: We have read the NeurlPS Code of Ethics and confirm that our research complies with these ethical standards.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Please see the Appendix G.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
18

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [NA]
Justification: This paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All the thing in this paper credited and are the license and terms use explicitly mentioned and properly respected.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the Datasets.papersstt paperswi thcode. cww/ datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
19

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: This paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
20

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: The method development in this paper does not involve LLMs any compo- nents.
Guidelines:
The answer NA means that the method development in this research does involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should be described.
21

---

Appendix
A Detailed Experimental Settings
Datasets. We evaluate our model on both discriminative and generative datasets, as listed below. (a) MMVP 442 evaluates recognition and reasoning performance across nine categories of basic visual patterns. (b) MMEval-Pro [3] assesses cross-modal understanding through triplet-based object and attribute recognition in natural images. (c) VMCBench [6]] use adversarial distractors test fine-grained discriminative ability across diverse tasks such as commonsense reasoning, image-text matching. All three datasets (a-c) adopt accuracy as the evaluation metric. (d) Bingo [5] evaluates bias and interference hallucinations, with GPT-4o used to score hallucination severity and response quality. (e) MMHAL-Bench 39 evaluates model capabilities beyond object hallucination, with GPT-4o used to assess hallucination rate and response informativeness.
Implementation Details. We select nine representative multimodal reasoning models evaluate their hallucination performance on general vision tasks. We categorize these models into two major training paradigms: (1) the RL-only paradigm, where models trained solely via reinforcement learning, including LMM-RI I [33], MM-RI [17]. ThinkLite- VL[46) MM-Eureka [30], and Ocean- RI [ (2) the two-stage paradigm, combining supervised fine-tuning (SFT) with reinforcement learning, including Vision-RI [15,, R1-One Vision
and Curr-ReFT [8. All models post-trained Qwen2.5-VL-3B or Qwen2.5-VL-7B, which are used baseline models.
B Reasoning Models Attention- Analysis
B.1
Visual Attention Heatmap
Figure T0 compares the visual attention distribution between multimodal reasoning models and their corresponding non-reasoning models. The results indicate that, compared to non-reasoning models, reasoning models exhibit weaker focus key image details, with attention more dispersed across other regions of the image. Specifically, reasoning models display greater degree of attention dispersion at lower layers, and their attention is not concentrated on critical areas of the image. In contrast, non-reasoning models demonstrate more precise visual grounding. For instance, as shown in Figure IO the attention maps of the non-reasoning model at layers 10 and 15 consistently focus on the target object, the white mouse, highlighting its sustained attention on the target.
Loyer
Layer
Layer 15
Image?
Recssming
Non- Reasoning
Figure 9: Case Study 1: Attention Heatmap in Counting Tasks.
Loyer 5
Loyer 10
Loyer 15
white mouse on right black keyboard?
Reasoning
Non- Reessninn
Figure 10: Case Study 2: Attention Heatmap in Object Localization.
22

---

B.2 More Examples of the Impact of Reasoning Length on Visual Perception Degradation
This section presents an additional example, including visual task involving counting, comparing the results of reasoning models and their corresponding attention maps under different reasoning lengths. is evident that an excessively lengthy reasoning process causes the model to disregard the visual information inherent in the image, instead relying more heavily on prior linguistic knowledge. In Figure I1 the attention maps clearly show that, under over reasoning conditions, the model's attention shifts more towards the instruction tokens following the image tokens, particularly towards the latter part of the instruction. This suggests that prolonged reasoning reduces the model's focus on the visual information, leading it to depend more the guidance provided by the linguistic instructions.
Overthinking
mony traffic lights there in the image?
Overthinking
Normal Think
lighty the efersenics hes then koanel on the left ees of the mage, righ apparrt ro he oxparin cortroling ourtte trfffic Now oaag ougerts that the teaf liaa vollle Bcood on thet reviinng traffic darinct dinetions o/taaan This
trafffic lights
Normal Think
stheke Besed on the see that there four traff cl lights. The
light teo thrrigh
mage seci to be part
liay centrol
offierest dreationa
lights.
Trege
Targe
Figure 11: Attention shift in the reasoning model under different reasoning length.
B.3 Error Analysis
In this section, we further focus on the error rates of multimodal reasoning models and non-reasoning models across different problem types, conducting a statistical analysis to compare the differences between the two. Figure 12a presents the error type statistics for the Bingo benchmark samples. By combining GPT-4o evaluations with manual inspection, we analyze the reasoning process and final answers of the reasoning model determine whether errors stem from reasoning or perception. If the model's errors arise from both reasoning and perception, and reasoning" errors. The statistical results indicate that the proportion of perception errors reasoning model decreases, with more errors originating from the reasoning process. This suggests that the reasoning model does not completely fail to interpret the image information, but rather diminishes its focus on perceptual information during reasoning. The evaluation results in Figure L2p further confirm this phenomenon: the overall error rate of the reasoning model is higher than that of the non-reasoning model, with more prominent proportion of errors coming from reasoning.
classify them as "perception
MYMP Halaciation
Mihaai Falucination
NWVPP Recooring
Mihhalu Reeooning
Penception Reasoning
Perception Error
Reasoning Error
Ervor Proportion Dutaset
80-
12%
6-
22%
30%
58%
28%
Non- Roi-RRosonnng
Ressoning
Qwen2,5-V1.
11-Onevision
Ocean-RI
MM--Fareka
Figure 12: Error type distribution and error proportions across reasoning and non-reasoning models in Bingo benchmark. (a) Pie charts showing the distribution of perception and reasoning errors for non-reasoning and reasoning models, with the breakdown of perception error, reasoning error, and combined perception & reasoning errors. (b) Bar chart illustrating error proportions in the MMVP and MMhalu benchmarks, comparing hallucination and reasoning errors across reasoning models.
23

---

C Reasoning Length Control
C.1 Comparison of Three Reasoning Length Control Straregies
In the manuscript, we have thoroughly explored three methods: Token Budget Forcing, Test Time Scaling, and Latent State Steering. The first two methods directly control the model's s reasoning length by using fixed-length truncation or soft expansion of the reasoning length, ensuring dynamic expansion within a predefined thinking length range. However, the limitations of Token Budger Forcing and Test Time Scaling are that they can only control the model's 's reasoning length to shorten or lengthen, lacking flexibility for more nuanced adjustments. In contrast, Latent State Steering introduces tuning coefficient a, allowing more flexible control over the model's 's reasoning length. By adjusting the value of at we can effectively quantify changes the extent of reasoning. All of our Latent State Steering experiments are dynamically adjusted within the range of a E [-0.15, 0. 0.1. Furthermore, in the subsequent RH-bench calculation of RH-AUC, it is precisely due to the flexibility of the Latent State Latent State Steering Strategy that we apply it to dynamically regulate the reasoning length and perform further quantification.
C.2 Model Performance Variation of the Latent State Steering Length Control Strategy
Figures 13 presents the visualization of performance variations for different models under the Latent State Steering strategy, with a ranging from [-0.15, 0.15). In Figure 13 the star symbol represents the performance under the base condition. It is clearly observed that the variation in reasoning length shows that the optimal intervals for reasoning models differ between reasoning and hallucination tasks, with both exhibiting non-monotonicity.
MathViata
MathVisio
MMBale
MMVP
Helltectont toatten RRAe
65
RI-Cne
Acc (%))
70
Acrerary
80
0.6
Accuraty (%6)
50
8
20
00.
4.15 -:65 0.65
0.15
0.15
Thinking Langh
Thinking
Thinking
This Length
MathVists
Visto
MMMisll
MMVP
dlatectaaiin Mate
Think
Accurne (N)
Arruaay %
0.5
Accuray (%)
60
8
04.
-0.15 .0.1 0005 0.8 (1 0.15
4015
.01
0..0 0
0C5
611
0.15
4.35
-01 15.55 5.05
0.35
0.15 -0. -C.05 0.05 03 0.5
Thinking Length
ThinkingLesg
Thinking Length
Thinkinn Lenpth
MathViste
Vision
MM
MMVP
Cination Rate
65
Occan-31
Accuracy (%)
8
0.6
Accuracy (%)
50
Accurary
8
85
0.4
40
-0.15 -L1 a.0 0.5 ILI 0.15
-01 4115
41ts 0
0C5
6.1
0.15
4.15
-01 -.55 0.05 0.15
40.1 -0. -0-0
0.05 0J 0.15 This king Length
Thinking Langh
ThinkingLonghh
Thinking Length
MathViste
Mathh Vision
MMMall
MMVP
Open LLTiake
Ascurerr (%))
Accurayy 2
Hnltoroaattev Note
Ascuracy (%))
0.6
100
8
0.4
65
4015 ILJ
0.15
4.15
-0.05 0.05 0.15
4.15
-0.05 0.03
Thinking Length
Thinklng Length
Thinking Lenght
Khinking Length
MathVVsts
Ma Vision
MMVP
Balleehenton Mate
65
MLM- Earcka
Aesunns (S)
8
%
2.5
Assumay (%0)
16
8
%
.015 0.1 0.05 3. 0.05 0. Thinking Langth
0.15 4.35
011
4005 0.
0.55
0.1
0.15
4.15
-2.1
3,05 0.1 Thinking Length
4.65
0.35 0.15 0.1 4605 0 0.05 03 0.15 Thinking Langth
Loinking..ongh
Figure 13: Model performance variation of the Latent State Steering strategy. The star symbol represents the original thinking length of the model without steering or test-time intervention.
24

---

D More Examples from RH-Bench
In this section, we present samples from different tasks and question types in the RH-Bench benchmark. As shown in Figures [4, and 14p, we display samples of open-ended responses and multiple-choic questions for the visual perception task. Additionally, Figures [5p and [1pp showcase samples of multiple-choie questions and open-ended responses for the visual reasoning task. The focus of the questions differs across tasks. For instance, the visual perception task typically emphasizes image content recognition and understanding, whereas the visual reasoning task places more focus on the ability to draw conclusions from the image. To ensure the aceuracy of the evaluation, we select samples that maintain much independence between tasks possible, minimizing interference between different tasks. This independence allows each task to reflect the model's performance in specific task type, thereby providing more accurate assessment of the model's 's performance across various tasks and its ability to balance performance between different task types.
How many feddy bears are stors?
What can you see in the Aesreflaction from the sunglosses?
Ia the elephant's is trunk rased or insered? Chaioss: (A) Roited (8)
whore the tay relotive the dag"
Ovvissn cantnd and
facing right. (0) the
ong is rare,
is fexing right. directly in frent of the
nane the doee rase facing lift (D) above the
doy's head and mose pointing bechwaadd
(a)
(b)
Figure 14: Examples of visual perception in RH-Bench.
servef pai epgnr de
Qu Quaman wi paried the nember rul rine
Jocard 55,000 mest Besuse ahs
conautt 9
ocer
fire
employece is the meximue? (helear/A)) 21 (9)
r5 your okd ole concemedielnn the los of the
ene netter-aate a mny dutee of aoud tris white odee are there in totel
spairs sa shou the faame and ther relidd
zun 21 (D Mar 21 (00 21 Air 21
inartmont's a ihe What ane zeretise to rite
the fyrm. numbere of poirt shave oe the two calonez surt focup
reture
ieatheet Galsiits
manent
neture mnteeaeenn cnclictle
Inotees
Smak ancuni
Invoumcci Bic (clan))))
15,903
18
13,003
Figure 15: Examples of visual reasoning in RHI-Bench.
25

---

E
Visualization Results
This section presents the responses of reasoning models and their corresponding non-reasoning
models across different visual tasks. As shown in Figures T6 and 17] in these error examples, it evident that reasoning models typically exhibit errors during the reasoning process, leading to inaccurate reasoning results and, consequently, the emergence of hallucinations in the final answers.
Non- Non-Reasoning Model
The sign large, green, stylized letters that read "PENARTH PIER, fellowed "BUIL T 1894" underneath. layout and font suggest it's title landmork sign. also notice smaller logo on the bottom right which includes the words "VALE OF GLAMORSAN LEISURE," indicating may be part local council tourism board Given the context, this is likely a historical pier with some heritage value.
Is the word in the logo penarth pier built 1894"?
Reasoning Model
PENARTH PIER
<hinkk>
Okay, so need to figure out the word logo "Penorth Pier Built 1894, Let me start by looking directly ot the image itself. As seen in the image, there is sign that reads "Penarth Pier" with 'Built 1894" undernatth it. The text green white background, and there's s also logo for "Volley Glamorgon Leisure." However, the question specifically asks about the word the logo, not the entire sign. logo mentioned stylized design with a green leaf motif. This logo is separate from the main sign that mentions Penortt Pier and its construction date, Therefore, the word the logo likely nome "Valley Blamoryan Leisure,"
1834
part the Valley Blamorgon
branding, which includes
"Penorth Pier Built 1894, So, putting together, the word the logo isn t Penorth Fier Built 1894"; it's something else. </think No. The word the logo "Valley Glamorgan Leisure," not "Penorth Pier Built 1894."
Figure 16: Hallucination example 1 of reasoning models in visual tasks.
Non- Rei--Reasnnn Model
clesely examined the buildings the image. The ones on the left are mastly made of red brick and white walls, while those the right appear to covered in light gray beige materials.
There yellow brick building in the image?
structure near the center-night looks more like has light brown or cream-colored surfece, not the distinct tone or texture of yellow brick. Therefore, building the image clearly appears be made yellow brick.
Reasoning Model
<think> Okky,, so need to determine there's s a yellow brick building in image based on the provided content. Let me go through this step by step. First, I'll examine the image scene with multi-story buildings These buildings have a mix of architectural styles and colors-some are white with red accents, others are red brick with white trim. There's also modern building with flat roof wooden facade visible further down the street. Next, I'll look of the bounding boxes provided, One box is labeled aa yellow brick building with coordinates xmin: 208, ymin: 175, xmax: 346, ymaa.: 292, This indicates that within these specific coordinates, there is indeed yellow building present
itself. The image
urban
left side.
image. Therefore, based bounding box information, there yellow brick building in the image. </think> Yes, there yellow brick building in the image. The image shows yellow brick building within the specified coordinates, confirming presence.
the image the
Figure 17: Hallucination example 2 of reasoning models in visual tasks.
26

---

CH-Bench Evaluation Prompts
RH-Bench Visual Reasoning MCQ & Open-Ended Ended Prompt
You impartial evaluator assessing the correctness of model's answer multiple-choice question.
MCQ:
Question: {question]
Choices: {choices)
Model's Answer: (model answer)
Correct Answer: fground truth}
Please evaluate whether the model's answer is correct by considering:
1. Whether the model's answer matches the correct answer exactly (e.g.. same option letter). If the model's answer a value, whether matches the value of the correct option.
3. Whether the model's reasoning (if provided) supports its answer.
Your response should be JSON object with the following structure:
"s__corrctt;: <true false>,
"reason": "<brief explanation your evaluation>" "
Open-Ended: You impartial evaluator assessing the correctness of model's answer to multiple-choice question.
Question: (question)
Model's Answer: (model answer)
Correct Answer: (ground truth)
Please evaluate whether the model's answer is correct by considering:
Whether the model's answer matches the correct answer exactly (c.g.. same option letter). 2. If the model's 's answer is a value, whether it matches the value of the correct option. Whether the model's reasoning (if provided) supports its answer.
RH-Bende Visual Perception MCQ Prompt
Please evaluate whether the model's answer to the multiple-choice question is correct by considering: 1. Whether the model's answer matches the correct answer exactly (same option letter).
2. If the model's s answer is a value, whether it matches the value of the correct option.
Whether the model's reasoning (if provided) supports its answer.
Question: ()
Options: {}
Correct Answer: {}
Model's Answer: {
Your response should be JSON object with the following structure:
:s__corrett": <boolean>,
"reason": "<explanation of your evaluation>". Smodel_answer_evvi "<the extracted answer from the model's response>"
27

---

RH-Bench Visual Perception Open-Ended Prompt
Please act impartial and objective judge evaluate the presence and severity of hallucination in the response provided by a Large Multimodal Model (LMM) to the user question. Hallucination, this context, refers response that includes information not present or implied in the image or previous conversation. hallucination could be false claim about an object, action, emotion, or any other detail not grounded in the image.
situation where the LMM generates
Your task is to determine whether hallucination exists and, if present, to categorize its severity false information. Use the provided image contents, question, standard human-generater generated answer, and L.M response to make your judgment.
based on the extent and impact of the
Evaluation Guidelines:
Compare the LMM response with the standard auman-generated answer. Identify any claims in the LMM response that are not supported by the image question. Assess the severity of hallucination based on the nature and extent of the false information. The standard numan-generatie answer may not describe all objects details comprehensively. whether the LMM introduces false claims rather than omitting details. Detailed reasoning or analysis by the LMM that is consistent with the image contents is acceptable and should not be considered hallucination.
focus
Task: Evaluate the LMM response for hallucination based on the provided image contents, question, and standard human- -ennanngeeeeeedd answer. Provide a brief explanation of your analysis, identifying any false claims and their severity. Then, assign a rating from the following six levels:
Rating: 5: No hallucination (response is fully consistent with image contents).
Rating: 4: Minimal ballucination (minor, negligible false detail that does not significantly alter the response's accuracy, c.g., slight misdescription of color or background).
Rating: 3: Mild hallucination (small false claim that adds unsupported but non-critical detail, e.g., mentioning a minor object or attribute not present).
Rating: 2: Moderate hallucination (clear false claim that affects the response's 's accuracy, e.g.. incorrect object count or unsupported environmental detail).
Rating: I: Severe hallucination (significant false claim that misrepresents key elements, e.g.. entirely wrong objects or actions).
Rating: O: Extreme hallucination (response is largely entirely detached from the image contents, with multiple or critical false claims).
Question {
Ground-Truth Answer ()
LMM Response to Evaluate [)
Your response should be JSON object with the following structure:
Phallucination_score <integer from 0 to 5>,
"reason": "<brief explanation of your evaluation>"
28


Reviews:
[{"review_id": "iGAj6Ts2fQ", "reviewer": "Reviewer_kC9o", "summary": "This paper provides a systematic analysis of the phenomenon that as multimodal reasoning models enhance their reasoning abilities, their visual grounding capabilities deteriorate and hallucination issues become more pronounced. By comparing models of different scales and training paradigms, the authors show that longer reasoning chains make models increasingly rely on language priors at the cost of visual evidence, leading to higher hallucination rates. To address this, the paper proposes the RH-AUC metric and the RH-Bench diagnostic set to systematically quantify the trade-off between reasoning and hallucination, and provides an in-depth analysis of how various factors influence this balance.", "questions": "The conclusions drawn in this paper are meaningful, but the current experiments lack evaluation across more model series and sizes (see W2 & W3), raising concerns about the reliability and generalizability of the findings. Additionally, if the authors could further explore different RL methods and examine whether specific types of hallucination are linked to particular data or tasks (e.g., whether hallucinations are more common in counting scenarios), it would significantly enhance the value of this work (W4).", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "***Strengths:***\n1. The paper introduces a novel metric (RH-AUC) and a comprehensive benchmark (RH-Bench) for systematically quantifying the trade-off between reasoning and hallucination in multimodal reasoning models.\n2. Through extensive experiments and attention analysis, the study reveals and substantiates the important phenomenon that enhancing reasoning abilities in MLLMs can increase visual hallucinations, especially with longer reasoning chains.\n3. The paper evaluates different training paradigms and model scales, demonstrating that the reasoning-hallucination trade-off is a general phenomenon, and provides useful insights for future model development.\n\n***Weaknesses:***\n1. The size of the RH-Bench benchmark is limited, which raises concerns about whether it is sufficient for a comprehensive evaluation of reasoning-hallucination trade-offs across diverse tasks and settings.\n2. The experiments are limited to a single model series (mainly Qwen2.5-VL), with little evaluation on other mainstream multimodal model families.\n3. The study does not include a sufficient range of model sizes and architectures, making it difficult to assess the generality of the conclusions across diverse model types.\n4. The paper does not provide a more detailed analysis of hallucination types or investigate whether different categories of hallucination are related to specific data characteristics, which could potentially lead to a deeper understanding of the problem.", "was_revised": true, "final_justification": "I appreciate the authors' efforts, and they have addressed my concerns during the rebuttal period.", "rebuttal": "We appreciate the reviewer's thoughtful comments and recognition of **our novel evaluation metric and benchmark, systematic analysis of reasoning-hallucination trade-offs, and comprehensive experiments across training paradigms**. We sincerely hope our response below can address your concerns.\n\n**(W1) Scale of RH-Bench and Effective Evaluation.**  Thank you for your insightful suggestion. The effectiveness of the evaluation provided by RH-Bench arises from its careful design, rather than solely relying on sample size. As illustrated in Figs. 10-11 of Appendix D, the non-monotonic trend in model performance with varying reasoning lengths observed in RH-Bench closely aligns with trends observed in datasets such as MathVista and MMhalu (Fig. 7 in manuscript), demonstrating the capability of the benchmark to reliably capture model behaviors. Furthermore, RH-Bench effectively distinguishes performance differences among models. This capability results from the design of task diversity, with reasoning tasks involving core domains such as math, visual common sense, and scientific questions, while perception tasks focus on detail recognition and object identification across both multiple-choice and open-ended formats. We plan to further expand the scale and dimensions of RH-Bench to facilitate more comprehensive and in-depth model evaluations.\n\n**(W2 & W3) Generalizability Verification Across More Model Families.**  Thank you for your valuable feedback. In response to W2 and W3 on model size and framework, most existing multimodal reasoning models often adopt Qwen2.5-VL as a backbone model and fine-tune on it. Therefore, the set of models we evaluated covers the same model series. To further demonstrate generalizability, we conduct additional experiments on recently released reasoning models and baselines with different model series: Qwen2-VL (7B), InternVL2.5 (8B, 38B), and Qwen2.5-VL (32B, 72B). These models are tested on four perception benchmarks and the RH-Bench. As shown in Table 1, the results from these broader models are consistent with our findings, providing strong evidence of the general applicability of our study. We will include the new results in the revision of the paper.\n\n**Table 1: Performance of recently released multimodal reasoning models on perception benchmarks and RH-Bench.**\n| Method             | Train Setting | Training Data | Training Data | MMVP     | MMEval-Pro | VMCBench | MMHALU | MMHALU | RH-Bench |\n|--------------------|---------------|---------------|---------------|----------|------------|----------|--------|--------|--------|\n|                    |               | Perc.         | Reas.         | Acc ↑    | Acc↑        | Acc↑      | Score↑ | Hallu↓ | RH-AUC↑    |\n| **Qwen2-VL-7B**        | Base          | -             | -             | 45.7     | 75.0       | 82.7     | 3.48   | 0.36   | -      |\n| R1-VL [1]          | SFT+RL        | 21k           | 40k           | 43.8     | 73.7       | 81.5     | 3.40   | 0.39   | 0.47   |\n| **InternVL2.5-8B**     | Base          | -             | -             | 45.0     | 74.6       | 81.5     | 3.52   | 0.35   | -      |\n| MM-Eureka [2]      | RL            | 19k           | 38k           | 44.2     | 72.5       | 80.0     | 3.48   | 0.38   | 0.53   |\n| **InternVL2.5-38B**    | Base          | -             | -             | 49.5     | 78.0       | 83.3     | 3.57   | 0.33   | -      |\n| MM-Eureka [2]      | RL            | -             | 9k            | 47.6     | 77.2       | 81.6     | 3.53   | 0.35   | 0.64   |\n| **Qwen2.5-VL-32B**     | Base          | -             | -             | 50.4     | 78.5       | 84.0     | 3.63   | 0.32   | -      |\n| MM-Eureka-Qwen [3] | RL            | -             | 17k           | 48.5     | 77.9       | 82.8     | 3.60   | 0.34   | 0.63   |\n| Vision-R1 [4]      | SFT+RL        | 60k           | 10k           | 47.8     | 77.0       | 81.9     | 3.56   | 0.36   | 0.60   |\n| **Qwen-2.5-VL-72B**    | Base          | -             | -             | 51.0     | 79.3       | 86.1     | 3.70   | 0.30   | -      |\n| VL-Rethinker [5]   | RL            | 6k            | 14k           | 49.7     | 77.8       | 84.9     | 3.67   | 0.33   | 0.68   |\n| Vision-R1 [4]      | SFT+RL        | 60k           | 10k           | 50.8     | 78.0       | 84.8     | 3.63   | 0.37   | 0.65   |\n\n**(W4) Analysis of Hallucination Across Seven Data Types.**  Thank you for your thoughtful feedback. We agree that performing this analysis could lead to a deeper understanding; therefore, following the suggestion, we conduct an analysis comparing the average hallucination rates of reasoning models to baseline models across 7 categories of hallucination. As shown in Table 2, the results indicate that most reasoning models exhibit higher hallucination rates across different hallucination types, aligning with our previous finding. Specifically, reasoning models demonstrate comparatively higher hallucination rates in tasks like existence, color, counting, OCR, and Chart & Doc, and the gap is larger on OCR and Chart & Doc, which might require more advanced visual capability.\n\n**Table2: Hallucination rates (%) of reasoning models across different hallucination types.**\n| Method                 | Presence | Count | Position | Color | OCR | Chart & Doc | Others |\n|------------------------|----------|-------|----------|-------|-----|-------------|--------|\n| Curr-ReFT-3B           | 30       | 35    | 23       | 12    | 45  | 36          | 25     |\n| Vision-R1-3B           | 32       | 33    | 25       | 9     | 43  | 40          | 24     |\n| **Qwen2.5-VL-3B**          | 28       | 34    | 17       | 12    | 43  | 34          | 23     |\n| R1-OneVision-7B        | 28       | 31    | 20       | 5     | 38  | 33          | 22     |\n| ThinkLite-VL-7B        | 20       | 32    | 19       | 10    | 36  | 30          | 20     |\n| **Qwen2.5-VL-7B**         | 23       | 30    | 15       | 7     | 30  | 27          | 18     |\n| MM-Eureka-Qwen-32B     | 19       | 26    | 13       | 6     | 29  | 25          | 15     |\n| **Qwen2.5-VL-32B**         | 18       | 23    | 10       | 5     | 26  | 22          | 10     |\n| VL-Rethinker-72B       | 19       | 22    | 13       | 6     | 28  | 20          | 12     |\n| **Qwen2.5-VL-72B**         | 16       | 20    | 8        | 5     | 25  | 19          | 9      |\n\n\n\n[1] *R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimization*\n\n[2] *MM-Eureka: Exploring visual aha moment with rule-based large-scale reinforcement learning*\n\n[3] *MM-Eureka: Exploring the frontiers of multimodal reasoning with rule-based reinforcement learning*\n\n[4] *Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models*\n\n[5] *VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learning*"}, {"review_id": "RThWFXwWAp", "reviewer": "Reviewer_u6Et", "summary": "This paper focuses on the phenomenon that multimodal reasoning models tend to exhibit more hallucinations in perception tasks compared to non-reasoning models. Through empirical studies, they attribute this to reasoning models relying more on language priors and paying less attention to visual input, which causes hallucinations. Furthermore, the paper finds that the optimal reasoning intervals for minimizing hallucinations differs from that for maximizing reasoning performance, and existing metrics fail to capture the balance between hallucination and reasoning ability. To better evaluate this trade-off, the authors propose the RH-AUC metric and the RH-Bench dataset, providing insights based on results from multiple reasoning models.", "questions": "1. I recommend a deeper discussion on the soundness of the Latent State Steering operation.\n2. It would be beneficial to include additional empirical analysis of the training paradigms to provide more valuable insights into the training of multimodal reasoning models.\n3. A clearer explanation of the RH-AUC metric, along with a comprehensive analysis of its advantages, would greatly enhance the paper.\n4. The results presented in Figure 6a of the appendix appear somewhat counterintuitive: the reasoning model exhibits fewer perception errors but more reasoning errors compared to the non-reasoning model, which seems to contradict the observations highlighted in the main text. Could a plausible explanation for this discrepancy be provided?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "### Strengths\n1. The paper focuses on an important issue that multimodal reasoning models are more prone to hallucination. Through a systematic analysis across a range of advanced models, it offers valuable insights, including that models trained with SFT followed by RL are most susceptible to hallucination.\n2. The evaluation of multimodal reasoning models should account for both reasoning ability and hallucination severity. This work introduces a metric designed to reflect both factors.\n3. The paper is well-written and clearly structured, making it easy to follow.\n\n### Weaknesses\n1. **Limited Novelty in Attribution Analysis.** Prior work, such as [1, 2], has already highlighted the over-reliance on language priors as a key contributor to hallucinations. The observations in this work—such as weak visual attention and declining visual focus over longer reasoning chains—seem to be natural extensions of these findings rather than fundamentally new insights.\n2. **Lack of a Technical Contribution.** While the paper raises an important and timely question, it does not propose a concrete technical method or solution, which constrains its contribution.\n3. **Limited Empirical Support for Training Paradigms Claims.** The paper’s claim that SFT leads to rigid imitation reasoning and redundant steps would benefit from more direct empirical validation.\n4. **Concerns About the Methodological Soundness of Latent State Steering.** Since the outputs are generated from an intervened latent state rather than the model’s natural reasoning process, it remains unclear how well they reflect the model’s true capabilities. This raises questions about the validity of key analyses (Figure 6 and 7) and the RH-AUC metric that rely on this technique.\n5. **Interpretability and Effectiveness of the RH-AUC Metric.** The RH-AUC metric correlates strongly with both perception and reasoning performance. This suggests it may reflect overall performance rather than specifically measuring the balance between reasoning and perception. More analysis is needed to demonstrate its advantage over using perception and reasoning accuracy directly.\n\n[1] Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding\n[2] Paying More Attention to Image: A Training-Free Method for Alleviating Hallucination in LVLMs", "was_revised": true, "final_justification": "While I still regard the technical and conceptual contributions as somewhat incremental, and believe that the conclusions regarding the training paradigm would benefit from further substantiation (I emphasize this point as I consider it a key aspect of the paper’s potential value), I appreciate that the additional experiments—particularly those presented in the responses to other reviewers—have further strengthened the overall quality and contribution of the work. Accordingly, I am raising my score to 4.", "rebuttal": "We thank the reviewer for the valuable feedback, and for recognizing **the importance of our proposed problem, the insightful findings provided by our systematic analysis, and the clarity and readability of our paper’s presentation**. We sincerely hope our response below can address your concerns.\n\n**(W1) Discussion with Relevant Works.**  Thank you for highlighting these relevant works. We will include the suggested discussion in the revised manuscript. While we indeed follow the attention analysis approach from prior work, the focus of our paper is different. We are not attempting to explain the general source of hallucination in language models; rather, we aim to explain how hallucination in reasoning language models is amplified. Specifically, our attention analysis demonstrates that reasoning models produce longer reasoning chains, which consequently leads to reduced attention toward visual inputs.\n\n**(W2) Concrete Technical Contribution.**  Thank you for bringing up this point. We indeed proposed several techniques to balance the hallucination and reasoning ability of multimodal reasoning language models. Specifically, the **Latent State Steering** method (mentioned in Sec. 4.1 of the manuscript) is designed to mitigate hallucination while maintaining reasoning ability (see detailed analysis in table3 below) by dynamically adjusting latent representations to control the reasoning chain length. As demonstrated in Fig. 9–11 in the Appendix, our approach effectively reduces hallucinations without sacrificing the original reasoning performance of the model.\n\n**(W3 & Q2) Empirical Support for Training Paradigms Claims.**  Thank you for your valuable suggestion. We agree that some more direct validation would make the argument stronger, and we demonstrated some preliminary analysis in the paper: in Fig. 8b, we demonstrate that SFT+RL may increase reasoning length. Moreover, we illustrate differences in reasoning patterns between these paradigms through a case study presented in Fig. 8c. To further investigate the difference in reasoning behavior between SFT and SFT+RL models, we compare the average token length and the frequency of reasoning-related vocabulary in responses from two model paradigms. As the results in the Table 1 and 2 indicate, SFT+RL models generate significantly more redundant responses than RL models, which aligns with our findings.\n\n**Table1: Average response token length.** \n| Benchmark  | Baseline (k) | SFT + RL (k) | RL (k) |\n|------------|--------------|--------------|--------|\n| MathVision | 0.70         | 4.85          | 0.81    |\n| MathVista  | 0.35         | 3.23       | 0.38   |\n| MMVP       | 0.20         | 1.80         | 0.23   |\n| MMHalu     | 0.13         | 1.72         | 0.16   |\n\n**Table2: Frequency of reasoning related words evaluated on MathVision.** \n| Word              | Baseline | SFT + RL | RL   |\n|-------------------|----------|----------|------|\n| `\"wait\"`          | 850      | 3205     | 2353 |\n| `\"check\"`         | 783      | 2040     | 1780 |\n| `\"alternative\"`   | 566      | 1204     | 800  |\n| `\"however\"`       | 620      | 1532     | 980  |\n\n**(W4 & Q1) Validation and Analysis of Latent State Steering.**  Thank you for this suggestion. We would like to clarify that our proposed steering method does not compromise the model's natural reasoning capabilities. Based on the results presented in Table 3–4 below, we evaluate the performance of three length control methods: Token Budget Forcing, Test Time Scaling, and Latent State Steering on RH-Bench. The results show that when controlling for the same length range, the differences in the impact of these methods on model performance in perception and reasoning tasks are minimal. This suggests that the observed performance variations are mostly due to the reasoning length, rather than the specific control method. Furthermore, our steering method uses the hyperparameter $\\alpha$ to control output lengths, where setting $\\alpha=0$ allows the model to reason without intervention.\n\n**Table3: Accuracy (%) comparison on RH-Bench across three reasoning length control methods: Token Budget Forcing, Test Time Scaling, and Latent State Steering (LSS).**\n\n| Think Length |                |       Perception           |                   |                  |       Reason             |              |\n|--------------|--------------------------|--------------------|-------------------|--------------------------|------------------|--------------|\n|                   | Scaling                  | Budget             | LSS               | Scaling                  | Budget           | LSS          |\n| 50–150       | 54.8                     | 54.5               | 55.0              | 45.4                     | 45.2             | 45.5         |\n| 151–250      | 56.0                     | 56.2               | 56.5              | 46.3                     | 46.0             | 46.4         |\n| 251–350      | 56.5                     | 56.0               | 56.8              | 47.8                     | 47.5             | 48.0         |\n| 351–450      | 57.6                     | 57.3               | 57.8              | 48.6                     | 48.4             | 48.9         |\n| 451–550      | 55.8                     | 55.6               | 56.1              | 48.1                     | 48.3             | 48.6         |\n\n**Table4: RH-AUC scores across different models under three reasoning length control methods.**\n\n| Model                 | Scaling | Budget | LSS   |\n|-----------------------|---------|--------|-------|\n| LLM-R1-3B             | 0.462   | 0.464  | 0.465 |\n| Curr-ReFT-3B          | 0.473   | 0.470  | 0.473 |\n| R1-OneVision-7B       | 0.465   | 0.465  | 0.467 |\n| ThinkLite-VL-7B       | 0.525   | 0.523  | 0.526 |\n| MM-Eureka-Qwen-7B     | 0.550   | 0.550  | 0.552 |\n| MM-Eureka-Qwen-32B [1]   | 0.609   | 0.607  | 0.610 |\n| VL-Rethinker-72B [2]      | 0.635   | 0.632  | 0.635 |\n\n\n**(W5 & Q3) Interpretability Analysis and Evaluation Advantages of the RH-AUC.**  Thank you for your insightful comments. We would like to clarify that RH-AUC is a metric designed to assess the interaction between perception and reasoning across varying reasoning lengths, reflecting an overall balance. This metric is obtained by plotting perception accuracy (y-axis) against reasoning accuracy (x-axis) at various reasoning lengths. As reasoning length increases, perception accuracy typically declines due to diminished visual attention (as shown in Figures 5–6 of the manuscript). Therefore, the area under this curve captures the overall balance between perception and reasoning.\n\nThe advantage of RH-AUC over traditional metrics lies in its ability to assess the interaction between perception and reasoning, rather than just measuring their individual accuracy. A simple average of perception and reasoning accuracy inevitably masks their critical interaction under various reasoning lengths. For example, a model with 0.8 perception accuracy but 0.2 reasoning accuracy, and another model with both scores at 0.5, would have the same average (0.5). However, the second model shows better balance. Traditional metrics fail to reflect this, while RH-AUC effectively evaluates the balance between perception and reasoning by testing performance across different reasoning levels.\n\n**(Q4) Further Analysis of Error Proportions in Reasoning Models.**  Thank you for bringing up this point. The distinction between reasoning and perception errors in Appendix Figs. 6a is as follows: perception errors occur at the initial stage of the response, showing wrong identification of visual content (e.g., misidentifying three dogs as two in an image). In contrast, reasoning errors arise during the reasoning after the model has correctly perceived the content (e.g., accurately recognizing the spatial relationship between a red car and a tree but drawing an incorrect answer about whether the car is parked under the tree). The increase in the proportion of reasoning errors further supports our finding that the decline of attention with longer reasoning chains is the key factor exacerbating hallucination, rather than issues in early-stage visual information processing.\n\n[1] *MM-Eureka: Exploring the frontiers of multimodal reasoning with rule-based reinforcement learning*\n\n[2] *VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learning*"}, {"review_id": "UluAT24X54", "reviewer": "Reviewer_cecd", "summary": "This paper studies the hallucination observed in recent reasoning VLMs. The authors begin with a preliminary analysis showing that compared to the original VLMs, reasoning VLMs with reasoning chains tend to produce more perception-related hallucinations. Through attention-based analysis, they identify that reasoning models tend to reduce visual attention over time, which leads to the rise in hallucination. Further investigation reveals a non-monotonic relationship between reasoning length and  reasoning-perception performance.\n\nTo quantify the trade-off between reasoning and hallucination in VLMs, the authors propose: (1) RH-AUC, a metric that measures the perception accuracy as reasoning length varies; and (2) RH-Bench, a diagnostic benchmark designed to evaluate both reasoning and hallucination across diverse multimodal tasks. Experimental results show that larger models generally achieve a better balance between reasoning and hallucination, and that the types and domains of training data significantly influence the performance.", "questions": "## Questions \n\nIn addition to the concerns raised in the weaknesses section:\n- Have you considered incorporating larger VLMs (e.g., GPT-4) into your experiments to assess how their accuracy varies with different reasoning chain lengths? This could provide a broader perspective on the reasoning–hallucination trade-off.\n- In the results presented in Section 5, which length-control methods (e.g., Token Budget Forcing, Test-Time Scaling, Latent State Steering) were used to generate reasoning chains of different lengths? Does the choice of length-control method impact model performance on RH-AUC? Is it possible that a model achieves a low RH-AUC due to the control method rather than the model’s actual capabilities?\n\n## Writing\n- Line 67: missing space after the comma.\n\n\n*Note: I am currently leaning toward a borderline rejection. However, I am open to increasing my score if the concerns outlined above are adequately addressed during the rebuttal phase.*", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 2, "originality": 3, "strengths_and_weaknesses": "## Strengthes\n- This paper addresses a timely and important problem concerning hallucination in recent reasoning VLMs. The observations and conclusions presented may offer insights for future research in the field. The overall motivation and logical flow of the paper are clear and well-structured.\n- The introduction of RH-AUC and RH-Bench allows for a nuanced quantification of the trade-off between reasoning and visual grounding, enabling a more comprehensive evaluation of recent VLMs on this issue.\n- The paper conducts comparisons across multiple recent models with varying sizes (e.g., 3B and 7B) and training strategies (e.g., SFT and RL), offering insights that could benefit the development of future models.\n\n## Weaknesses\n\n- Some of the conclusions and analyses regarding data domains, training paradigms, and model performance are primarily observational, based on off-the-shelf models, without the support of rigorously controlled experiments (e.g., training models with varied data or training strategies). As a result, certain claims—such as “RL only outperforms SFT+RL” or “data domain matters more than data volume”—may be less convincing.\n- While RH-AUC enables detailed measurement of performance across reasoning chain lengths, it comes with high computational costs, potentially limiting its practical usability. Moreover, enforcing strict control over reasoning length can be rigid, especially given that current reasoning VLMs are not trained with explicitly controllable reasoning depth. For example, a model might naturally prefer shorter reasoning chains to achieve optimal answers. A potentially more realistic evaluation would allow models to generate multiple diverse reasoning chains per question, assessing performance without prescribing specific lengths.\n- A related concern is that more complex questions often elicit longer reasoning chains. However, RH-Bench does not appear to incorporate this factor into its design, which may limit its effectiveness in accurately evaluating model performance across varying question difficulties.\n- The current analysis is primarily conducted on models from the Qwen2.5-VL (as noted in the limitations), which may constrain the generalizability of the findings to broader classes of VLMs.", "was_revised": true, "final_justification": "The author responses have addressed most of my concerns. Overall, I consider the work interesting and relevant to a timely problem. One remaining question lies with W1: although the rebuttal includes comparisons across various VLMs to assess the influence of SFT and RL training, these models may differ in terms of training data, implementation details, and underlying paradigms. As such, the conclusions drawn may not be entirely robust.\n\nIn general, I appreciate the authors' efforts and will raise my score to a 4.0 as borderline accept.", "rebuttal": "We sincerely thank the reviewer for recognizing **our observations as timely, insightful, and important, our comprehensive quantitative evaluation, and our systematic analysis of the reasoning–hallucination trade-off**. We sincerely hope that our responses below address your concerns. \n\n**(W1) Empirical Analysis and Generalizability Validation.**  Thank you for pointing this out. We agree that our current analyses are based on off-the-shelf models, and while they offer valuable insights, they should be interpreted as observational rather than causal,  this is indeed what we have done in the paper where we show our findings suggest trends and raise hypotheses, such as “RL only outperforms SFT+RL”, we will make this even clearer in the revised version and warrant future controlled studies. To further demonstrate the generalizability of our findings, we conduct a comprehensive performance evaluation of recently released reasoning models. As shown in Table 1 of **qZ13 W1**, the results not only reveal a consistent trend of amplified hallucinations but also further support the finding that training data volume is not the sole determinant of model performance. As shown in Table 1 below, the performance comparisons across different training paradigms, along with the statistical analyses of reasoning length and reasoning-related terms presented in Tables 2–3, are consistent with the findings reported in our manuscript.\n\n**Table1: Performance comparison of two training paradigm models on multimodal benchmarks.**\n| Method                | Paradigms | MMVP        | MMEval-Pro  | VMCBench    | MMHALU | MMHALU | RH-Bench | RH-Bench  |\n|-----------------------|-----------|-------------|-------------|-------------|--------------|--------------|----------------|----------------|\n|                       |           | Acc ↑       | Acc ↑       | Acc ↑       | Score↑           |  Hallu↓            |    Perc.↑             |    Reas.↑            |\n| SophiaVL-7B [1]       | SFT+RL    | 45.0        | 73.0        | 80.2        | 3.25         | 0.45         | 58.7           | 52.3           |\n| ReVisual-R1-7B [2]    | SFT+RL    | 46.7        | 72.5        | 80.0        | 3.30         | 0.44         | 60.2           | 53.5           |\n| VLAA-Thinker-7B [3]   | RL        | 47.4        | 75.7        | 81.3        | 3.46         | 0.41         | 64.5           | 56.4           |\n| MM-Eureka-Qwen-7B [4] | RL        | 46.7        | 74.8        | 82.0        | 3.43         | 0.43         | 62.4           | 54.0           |\n\n**Table2: Average response token length.** \n| Benchmark  | Baseline (k) | SFT + RL (k) | RL (k) |\n|------------|--------------|--------------|--------|\n| MathVision | 0.70         | 4.85          | 0.81    |\n| MathVista  | 0.35         | 3.23         | 0.38   |\n| MMVP       | 0.20         | 1.80         | 0.23   |\n| MMHalu     | 0.13         | 1.72         | 0.16   |\n\n**Table3: Frequency of reasoning related words evaluated on MathVision.** \n| Word              | Baseline | SFT + RL | RL   |\n|-------------------|----------|----------|------|\n| `\"wait\"`          | 850      | 3205     | 2353 |\n| `\"check\"`         | 783      | 2040     | 1780 |\n| `\"alternative\"`   | 566      | 1204     | 800  |\n| `\"however\"`       | 620      | 1532     | 980  |\n\n**(W2) Explanation of RH-AUC Computation.**   Thank you for your insightful suggestion. We strongly agree with the view that models should be able to generate diverse reasoning chains per question, and our **Latent State Steering** method (Section 4.1) is specifically designed to achieve this capability. This method enables flexible control over diverse reasoning chain generation through a single tunable parameter $\\alpha$, dynamically adjusting reasoning length without fixed constraints. Compared to direct generation of diverse reasoning chains, which often yields limited length variation and narrow scope, our steering method achieves significantly greater diversity through $\\alpha$ adjustment while preserving the model's original performance (see detailed analysis in Tables 4–5). Appendix Figs. 9–11 show the performance under our steering method. Notably, this method requires only a single hyperparameter adjustment without introducing additional computational overhead.  \n\n**(W3) Relationship Between Reasoning Length and Problem Complexity.**  We thank the reviewer for raising this important point. It is true that, in practice, more complex questions may naturally elicit longer reasoning chains. However, RH-Bench is designed to decouple reasoning length from question difficulty in order to evaluate the model’s robustness to reasoning depth under controlled settings. To minimize confounding effects, we fix the question while varying only the reasoning chain length via intervention, allowing us to probe how well the model performs as reasoning length increases even for the same question. This design choice helps us isolate the impact of reasoning length itself, rather than entangling it with question difficulty. We acknowledge this limitation and have clarified this design rationale in Section 5, while also noting that evaluating models under naturally varying question difficulty remains an important direction for complementary future benchmarks.   \n\n**(W4) Diversity of Base Models.**  Thank you for your valuable feedback. We respectfully note that most current existing multimodal reasoning models often adopt Qwen2.5-VL as a backbone model and fine-tune on it. Therefore, the set of models we evaluated covers the same model series. To further demonstrate generalizability, we conduct additional experiments on recently released reasoning models and baselines: Qwen2-VL (7B), InternVL2.5 (8B, 38B), and Qwen2.5-VL (32B, 72B). Detailed results are shown in Table 1 of **qZ13 W1**.\n\n**(Q1) Reasoning–Hallucination Trade-off in Larger VLMs.**  Closed-source models like GPT-4 and O3 struggle to precisely control reasoning chain length, relying mainly on limited prompt engineering. In contrast, our systematic evaluation of advanced open-source reasoning models (32B to 72B) in Table 1 of **qZ13 W1** demonstrates that larger models typically achieve higher RH-AUC scores, indicating more robust performance.\n\n**(Q2) Length-Control Method Impact on RH-AUC Performance.**  Thank you very much for your thoughtful suggestions. The RH-AUC evaluation employs the Latent State Steering method, with its implementation details shown in Appendix D. Furthermore, we conduct comparative experiments on the RH-Bench benchmark using three length control methods to assess their impact on model performance. As shown in Tables 4–5 below, by analyzing the accuracy of perception and reasoning tasks and the RH-AUC metrics of different models, the results indicate that the performance differences among the methods on these tasks fluctuate around ±0.5%. Moreover, the RH-AUC metrics demonstrate high stability across all three methods. These results indicate that the reasoning length control methods only affect the output length without compromising the model's performance.\n\n**Table4: Accuracy (%) comparison on RH-Bench across three reasoning length control methods: Token Budget Forcing, Test Time Scaling, and Latent State Steering (LSS).**\n\n| Think Length Range |                |       Perception           |                   |                  |       Reason             |              |\n|--------------|--------------------------|--------------------|-------------------|--------------------------|------------------|--------------|\n|                   | Scaling                  | Budget             | LSS               | Scaling                  | Budget           | LSS          |\n| 50–150       | 54.8                     | 54.5               | 55.0              | 45.4                     | 45.2             | 45.5         |\n| 151–250      | 56.0                     | 56.2               | 56.5              | 46.3                     | 46.0             | 46.4         |\n| 251–350      | 56.5                     | 56.0               | 56.8              | 47.8                     | 47.5             | 48.0         |\n| 351–450      | 57.6                     | 57.3               | 57.8              | 48.6                     | 48.4             | 48.9         |\n| 451–550      | 55.8                     | 55.6               | 56.1              | 48.1                     | 48.3             | 48.6         |\n\n**Table5: RH-AUC scores across different models under three reasoning length control methods.**\n\n| Model                 | Scaling | Budget | LSS   |\n|-----------------------|---------|--------|-------|\n| LLM-R1-3B             | 0.462   | 0.464  | 0.465 |\n| Curr-ReFT-3B          | 0.473   | 0.470  | 0.473 |\n| R1-OneVision-7B       | 0.465   | 0.465  | 0.467 |\n| ThinkLite-VL-7B       | 0.525   | 0.523  | 0.526 |\n| MM-Eureka-Qwen-7B     | 0.550   | 0.550  | 0.552 |\n| MM-Eureka-Qwen-32B    | 0.609   | 0.607  | 0.610 |\n| VL-Rethinker-72B [5]      | 0.635   | 0.632  | 0.635 |\n\n**(Q3) Correct Formatting.**  Thank you for highlighting this. We have carefully reviewed the text and made the necessary corrections in the revised manuscript.\n\n\n[1] *SophiaVL-R1: Reinforcing MLLMs Reasoning with Thinking Reward*\n\n[2] *Advancing Multimodal Reasoning: From Optimized Cold Start to Staged Reinforcement Learning*\n\n[3] *SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Models*\n\n[4] *MM-Eureka: Exploring the Frontiers of Multimodal Reasoning with Rule-Based Reinforcement Learning*\n\n[5] *VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learning*"}, {"review_id": "s4VwZGDdpJ", "reviewer": "Reviewer_qZ13", "summary": "This paper investigates the trade-off between reasoning and visual grounding in multimodal large language models (MLLMs), showing that while extended reasoning chains enhance task performance, they often amplify visual hallucinations due to reduced attention to image inputs and increased reliance on language priors. To quantify this trade-off, the authors introduce RH-AUC, a novel metric that captures the balance between reasoning accuracy and hallucination across varying reasoning lengths. They also release RH-Bench, a diagnostic benchmark comprising 1,000 reasoning and perception tasks to evaluate model performance holistically. Using attention analysis, the study finds that hallucination increases with longer reasoning chains, but the relationship is non-monotonic and task-dependent. Moreover, models trained with reinforcement learning (RL-only) outperform those using supervised fine-tuning plus RL (SFT+RL) in maintaining this balance, and the quality and domain of training data are more influential than data volume in achieving robust visual grounding.", "questions": "1- Could the authors elaborate on the specific aspects of the R1-OneVision training paradigm that may limit its ability to balance reasoning and perception, despite its access to extensive visual perception data? For example, are there architectural constraints, insufficient multi-modal supervision signals, or imbalanced task objectives contributing to this outcome?\n\n2- Given that RH-Bench includes diverse tasks beyond mathematics, could the authors clarify whether the observed RH-AUC gains for math-trained models (e.g., MM-Eureka, MM-R1) reflect general improvements in perception-reasoning balance, or are primarily driven by performance on math-like subsets of the benchmark? Would domain-wise breakdowns of RH-AUC help clarify this?", "limitations": "The paper makes a strong and valuable contribution by introducing a benchmark and analysis framework for evaluating perception-reasoning balance in multi-modal models. The overall framing is thoughtful and well-executed. However, some limitations could be more clearly acknowledged. For instance, the choice of non-reasoning baseline is somewhat narrow, which may affect the generalizability of certain claims about grounding behavior. Additionally, a few interpretive points—such as how model training paradigms affect performance or how domain-specific data impacts general reasoning—would benefit from further clarification. These are not critical weaknesses, but addressing them would improve the transparency and depth of the work. Overall, the paper is solid and deserves acceptance.", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "### Strengths \n\n1- Strong and Actionable Conclusions: The paper draws clear conclusions from its analyses, showing that longer reasoning can lead to more hallucination and that the best reasoning length depends on the task. These findings offer useful guidance for designing and fine-tuning multimodal reasoning models.\n\n2- Strong Benchmark and Metric: The authors introduce RH-Bench and RH-AUC, which together offer a solid and practical way to measure how well models balance reasoning with staying grounded in the image.\n\n3- Clear and Easy to Follow: The paper is well-organized and clearly written, with helpful figures and examples that make even complex ideas, like reasoning length control, easy to understand.\n\n### Weaknesses \n\n1- Narrow Choice of Non-Reasoning Baselines: The paper primarily uses Qwen2.5-VL as the non-reasoning baseline across all comparisons, limiting the diversity of grounding-focused models in the study. This makes it unclear whether the observed hallucination trends generalize across other non-reasoning MLLMs such as LLaVA, or MiniGPT-4, which may have different visual grounding behaviors and could serve as stronger or more varied baselines.", "was_revised": true, "final_justification": "The auhtors addressed my concerns and thus I maintain my positive score.", "rebuttal": "We would like to express our heartfelt gratitude to the reviewer for recognizing that **our proposed conclusions are valuable and actionable, our benchmark and metric are solid and practical, and our presentation is well-organized**. We sincerely appreciate the insightful questions raised and have addressed them in detail below.\n\n**(W1) Broader Baseline Coverage.**  Thank you for this suggestion. During our research, we observe most current existing multimodal reasoning models often adopt Qwen2.5-VL as a backbone model and fine-tune on it. Therefore, the set of models we evaluated covers the same model series. To further validate our findings, we conduct a broader evaluation of newer reasoning models across various sizes and their corresponding non-reasoning counterparts. As shown in Table 1, the results align with our findings, further corroborating the universality of the hallucination trends and visual grounding behaviors proposed in this paper.\n\n**Table 1: Performance of recently released multimodal reasoning models on perception benchmarks and RH-Bench.**\n| Method             | Train Setting | Training Data | Training Data | MMVP     | MMEval-Pro | VMCBench | MMHALU | MMHALU | RH-Bench |\n|--------------------|---------------|---------------|---------------|----------|------------|----------|--------|--------|--------|\n|                    |               | Perc.         | Reas.         | Acc ↑    | Acc↑        | Acc↑      | Score↑ | Hallu↓ | RH-AUC↑  |\n| **Qwen2-VL-7B**        | Base          | -             | -             | 45.7     | 75.0       | 82.7     | 3.48   | 0.36   | -      |\n| R1-VL [1]          | SFT+RL        | 21k           | 40k           | 43.8     | 73.7       | 81.5     | 3.40   | 0.39   | 0.47   |\n| **InternVL2.5-8B**     | Base          | -             | -             | 45.0     | 74.6       | 81.5     | 3.52   | 0.35   | -      |\n| MM-Eureka [2]      | RL            | 19k           | 38k           | 44.2     | 72.5       | 80.0     | 3.48   | 0.38   | 0.53   |\n| **InternVL2.5-38B**    | Base          | -             | -             | 49.5     | 78.0       | 83.3     | 3.57   | 0.33   | -      |\n| MM-Eureka [2]      | RL            | -             | 9k            | 47.6     | 77.2       | 81.6     | 3.53   | 0.35   | 0.64   |\n| **Qwen2.5-VL-32B**     | Base          | -             | -             | 50.4     | 78.5       | 84.0     | 3.63   | 0.32   | -      |\n| MM-Eureka-Qwen [3] | RL            | -             | 17k           | 48.5     | 77.9       | 82.8     | 3.60   | 0.34   | 0.63   |\n| Vision-R1 [4]      | SFT+RL        | 60k           | 10k           | 47.8     | 77.0       | 81.9     | 3.56   | 0.36   | 0.60   |\n| **Qwen-2.5-VL-72B**    | Base          | -             | -             | 51.0     | 79.3       | 86.1     | 3.70   | 0.30   | -      |\n| VL-Rethinker [5]   | RL            | 6k            | 14k           | 49.7     | 77.8       | 84.9     | 3.67   | 0.33   | 0.68   |\n| Vision-R1 [4]      | SFT+RL        | 60k           | 10k           | 50.8     | 78.0       | 84.8     | 3.63   | 0.37   | 0.65   |\n\n**(Q1) Factors Influencing The Reasoning-Perception Balance of R1-OneVision.**  Thank you for the insightful question. The performance limitations of the R1-OneVision model, built upon the Qwen2.5-VL architecture, are not attributable to inherent architectural constraints of the base model. This is evidenced by the fact that numerous other superior reasoning models also utilize the same baseline. The internal attention mechanism analysis of the R1-OneVision model, as depicted in Fig. 5 of the manuscript, reveals a significant dilution effect on visual information during the reasoning process. This suggests a potential imbalance in the task objectives of R1-OneVision, with the optimization focus skewed towards enhancing reasoning capabilities. Furthermore, the adoption of the SFT+RL training paradigm without integrating additional optimization strategies likely contributes to an imbalance between reasoning and perception abilities.\n\n**(Q2) Domain-wise Analysis of RH-AUC Performance.**  Thank you for highlighting this. To investigate the RH-AUC improvement of math-trained models (e.g., MM-Eureka and MM-R1) on RH-Bench, we categorize the benchmark into math and non-math tasks and separately evaluate the accuracy and RH-AUC scores of each model. As shown in Table 2, the results demonstrate that MM-Eureka and MM-R1 indeed achieve higher RH-AUC scores in pure mathematics tasks, clearly reflecting their superior perception-reasoning balance capability within the mathematical domain. Notably, even on non-math tasks, the RH-AUC of MM-Eureka and MM-R1 surpasses that of some models with extensive training data, such as R1-OneVision. This suggests domain-specific training holds the potential to generalize to other domains, improving perception-reasoning balance, which aligns with our paper's findings.\n\n**Table2: Comparison of RH-AUC across math and non-math domains for different models.**\n\n| Method              | Math | Non-Math |\n|---------------------|------|----------|\n| MM-Eureka-7B        | 0.58 | 0.54     |\n| MM-R1-7B            | 0.59 | 0.55     |\n| R1-Onevision-7B     | 0.45 | 0.49     |\n| ThinkLite-VL-7B     | 0.51 | 0.53     |\n\n**(L1) Further Clarification of Interpretive Points.**  Thank you for your valuable feedback. Regarding the impact of training paradigms and data domains, we present the performance and RH-AUC of various advanced models in Table 1. These results clearly indicate that recent reasoning models no longer rely solely on larger datasets but rather prioritize carefully curated multi-domain data, which aligns with our analysis. Furthermore, most RL-only models exhibit superior performance. For these two training paradigms (RL vs. SFT+RL), our analysis in Tables 3 and 4 below conducts fine-grained statistics on response length and reasoning-related vocabulary. The results reveal a statistically significant tendency of RL models to generate more concise responses while avoiding excessive deliberation, which is consistent with the results in Figs. 8b–c of the manuscript. Moreover, Table 5 presents a more comprehensive comparison of the RL and SFT+RL training paradigms.\n\n**Table3: Average response token length.** \n| Benchmark  | Baseline (k) | SFT + RL (k) | RL (k) |\n|------------|--------------|--------------|--------|\n| MathVision | 0.70         | 4.8          | 0.8    |\n| MathVista  | 0.35         | 3.2          | 0.38   |\n| MMVP       | 0.20         | 1.80         | 0.23   |\n| MMHalu     | 0.13         | 1.72         | 0.16   |\n\n**Table4: Frequency of reasoning related words evaluated on MathVision.** \n| Word              | Baseline | SFT + RL | RL   |\n|-------------------|----------|----------|------|\n| `\"wait\"`          | 850      | 3205     | 2353 |\n| `\"check\"`         | 783      | 2040     | 1780 |\n| `\"alternative\"`   | 566      | 1204     | 800  |\n| `\"however\"`       | 620      | 1532     | 980  |\n\n**Table5: Performance comparison of two training paradigm models on multimodal benchmarks.**\n| Method                | Paradigms | MMVP        | MMEval-Pro  | VMCBench    | MMHALU | MMHALU | RH-Bench | RH-Bench  |\n|-----------------------|-----------|-------------|-------------|-------------|--------------|--------------|----------------|----------------|\n|                       |           | Acc ↑       | Acc ↑       | Acc ↑       | Score↑           |  Hallu↓            |    Perc.↑             |    Reas.↑            |\n| SophiaVL-7B [6]       | SFT+RL    | 45.0        | 73.0        | 80.2        | 3.25         | 0.45         | 58.7           | 52.3           |\n| ReVisual-R1-7B [7]    | SFT+RL    | 46.7        | 72.5        | 80.0        | 3.30         | 0.44         | 60.2           | 53.5           |\n| VLAA-Thinker-7B [8]   | RL        | 47.4        | 75.7        | 81.3        | 3.46         | 0.41         | 64.5           | 56.4           |\n| MM-Eureka-Qwen-7B [3] | RL        | 46.7        | 74.8        | 82.0        | 3.43         | 0.43         | 62.4           | 54.0           |\n\n[1] *R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimization*\n\n[2] *MM-Eureka: Exploring visual aha moment with rule-based large-scale reinforcement learning*\n\n[3] *MM-Eureka: Exploring the frontiers of multimodal reasoning with rule-based reinforcement learning*\n\n[4] *Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models*\n\n[5] *VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learning*\n\n[6] *SophiaVL-R1: Reinforcing MLLMs Reasoning with Thinking Reward*\n\n[7] *Advancing Multimodal Reasoning: From Optimized Cold Start to Staged Reinforcement Learning*\n\n[8] *SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Models*"}]

Author rebuttal:

Meta-review: This paper analyzes how longer reasoning chains in multimodal models amplify hallucinations by weakening visual grounding. It introduces the RH-AUC metric and RH-Bench benchmark to quantify this trade-off, and the experiments provide evidence and insights. While questions remain about benchmark scale and novelty as pointed out by R, the rebuttal addressed them with some broader results. Overall, this is a timely and valuable contribution AC thinks. AC recommend weak accept.
Decision: Accept (poster)